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Preface 


Since the business of the Experimental Technology Incentives Program (ETIP) was 
change, it is difficult to capture in words the dynamics of the program's 
process. Not even scholars can agree on what constitutes the change process or, 
even more importantly, how best to affect it. The ETIP program and its method- 
ology of change is perhaps best seen as an art form, and it is for that reason 
the program turned in its last days to professional reporters and writers to 
record its story. The report contained in these pages is not an evaluation of 
ETIP in any definition of that word. Instead, ETIP asked Ralph Hoar and 

Richard Hebert to summarize from the over 100 published ETIP project reports, its 
many evaluations over the years, and extensive interviews or discussions with 
actual participants and agency partners. The report is the product of that 
distillation and fairly represents the history of ETIP, its major accomplish- 
ments, and its lessons for the future. In the end, ETIP hopes its actions speak 
at least as loud as its words and that the art form of its methods will continue 
to be used and refined. 


= 


ACKNOWLEDGEMENTS 


Much like the work of ETIP itself, the preparation of this report was a 
partnership. The authors would especially like to thank Dan Fulmer for his 
encouragement and gentle guidance and Stephen Garrity for sifting through moun- 
tains of wheat and chaff to provide numerous documents that served as grist for 
this report. 

The authors are also grateful for the information and insights provided 
by ETIP’s first director Jordan Lewis and its final director Robert Parsons. 
Others who provided valuable guidance and consultation include Richard Penn, 


Gregory Tassey, Charles Hulick, Sumner Clarren, Theodore Schlie and 


Leonard Sawicki. 


Li 


TABLE OF CONTENTS 


ACKNOWL EDGEMENTS e e e e e e e s e e e e e e e e e e s e e e e 


EXECUTIVE SUMMARY e e e e e e e e e e se e e @ o e e e e e e e e 


I 


II 


LLL 


INTRODUCTION e e e e s e es e e e e e e e e e e e es e e e e 


HISTORY AND STRATEGY . 0 .« got feayneySe teypenpepees ce) diet <i fel ‘el eh te 


A. Background © @ .© .6 0 © ,@ ,0 4e@ 46 ,e 6 ,o “eliietfed iet¥et.eo Velie le 


B. Experimentation Strategy .« « « « © « © « « « «© « «© « « 


C Program Areas and ProjectS « « « « « « «© « «© «© « © «© « 


D Program Closure e e e e es e ® e e e e e e e e e e e e 


ACHIEVEMENTS -— ETIP AS CHANGE AGENT « « « © © «© «© © « «© «@ 


ve Regulation ‘at eee te te oe “es “a: te Ye. *e He Se “eo fe te Fe te *e He fe, "oe 


l. 


4. 


FRA/ICC -- Fresh Produce Rail Transport 
Deregulation (Project No. 31). « « « « « « «© « « « 


FDA -- Post-Marketing Surveillance of Drugs 
(Proj ect No. 88) e e e e se 6 e e e e e e e e e e e 


FCC --— Public Record Message Service Deregulation 
(Project Nos. 140 and 161) eo © ee © © © © © © © @ @ 


Marketplace and Government ResultS « « « « « o « « 


B. Procurement e e e e e e e e e e e e e e e e e e e e e e 


l. 


Life Cycle Costing (Project Nos. 36 and 63)- « « - 
Value Incentive Clause (Project No~- 64). « « « « « 
Performance Specifications .« « «6 « « « « « « « « « 
Veterans Administration (Project Noe 80) - ... .- 
Mission-Agency Strategy -« « « «© + » «© © ee «© « « 


Marketpiace and Government Results .« . « « « « « » 


iii 


III-3 


ILI-6 


III-11 


III-13 


III-15 


III-18 


ILI-19 


III-23 


III-27 


ILI-30 


III-31 


IV 


IHerorojnbie LWoeslheskels 0.6 6 GO 6 Oo ood 6 6 6 Go 6 6 oO 6 


4. 


Regulatory Impacts on Small Businesses 
(Project No. 28) e « © © © © © © © 6 ©@ © © @ -« /@' \s 


Venture Capital and SEC (Project Nos. 9 and 148). . 


Connecticut Product Development Corporation (Project 
Nos. oa. 78, and 136) «eee ee © e © © © © © © @ 


Marketplace and Government Results -« « « « « « « « e« 


Research and Development ~ « « « « « « «© « « © «© « © « e 


1. Integrated Utility Systems (Project Noe 67) . ... - 


?. Flame-Retardants for Polyester/Cotton Blends 


(Project Nos. 7, 97, 125, 126, and 134) . . . . « « .» 


3. Marketplace and Government Results «. . « »« « « « « « 


Conclusions eo 28©@ ee © © e© © © © © © © © © © © © © © © @ 


l. 


Pde 


Marketplace Perspective «+ »« « © « «© «© «© «© e © « « 


Government Agency Perspective . « « « « « »« « « « « 


ACHIEVEMENTS -- ETIP AS STUDENT OF CHANGE . «© «© « » © « @ « 


A. 


Generalizable Lessons eo eee ee © © © © © © © © © © «© 


Long-Term Commitment to Experimentation . . « « « -« 
Stakeholder Collaboration . unig TV eM ee ewe te 
Interviewing for Expectations . « « « « « « » « « » 
Protection of Proprietary Rights « - « »« « « « « e« » 
Top-Level Commitment - « « © « © «© © «© «© © © © © « « 
Recognition of Agency Needs «. « « « « « « « « « « « 


Real-Time Monitoring for Effects « « « « « « « « « « 


Program Area Lessons en F Se Mer es Tetra. see 


1. 


Regulation e ¢ &¢ oe fe"e "ss oe "eo 6 6 © @ © © ‘6 "8 © © 


-- Market Mechanisms vs. Command/Control Peas 


iv 


ITI=32 


ILT1I-34 


III-36 


III-40 
III-42 
III-42 


IIlI-43 


III-46 
III-49 
III-50 
III-50 
be 
Iv=-1 
IV-1 
IV-2 
IV—2 
IV=-4 
IvV—4 
IV-5 
IV-5 
LV=7 
IV-7 
Iv-/7 


IV-8 


Vv 


3. 


-- Regulatory Doubt Can Impede Technological 
BAVENCG® (er iets hee) Ma hee eheonnt eee aes 6! Se. e 


-- Technology Drives the Regulatory Game ... 
PFOCUTEMENUE «hs <i sucyis U6 bails feurct v's 6 6 6 » «6 
-- Product and Industry Devolopment Stage .. . 
-- Government as Major Purchaser and End User 


--— Government as Middleman or Broker .... . 


-- Sensitivity to Commercial Market Strategies and 


PeErcepelOnG esos gue. see uewiette. 6's) sLe «16s 6 


-~- Procurement Used in Tandem with Other Policy 
Tools * e © e e e e e e e e s e ® e e e ® e 


-- User Roles in Development and Use of LCC 
Models e td e 6 e e eo e e e e i) e e e e @ e e 


Economic Assistance « « « « © « « «© « «© « «© e e 
-- Industry Strategies and Government Policies 
-- Impact of Subsidies on Technology .... . 
-- Policy Options for Commodity Supplies ... 
-- Small Businesses’ Special Problems ... . .« 
Research and Development . « « « « « « « « « e« 
-- Technology Push vs. Market Pull « - -~ « e « 
-- Limited Value of Demonstrations .. . « « » 
-- Alternative R&D Strategies Sal ea em a ns 
-— Consortium Viability »« « « « « « « « « « « « 


ere Political Sensitivity ° e e ° e ® ® e e e ° 


CONCLUS IONS e e e ° e * e es e e e e e © e e e e e e e e 


Ae 


WME AKT OE UES us sicu © Ti entee oral ws), st ost le sil oL elle kab ea. 8 


le 


No Overall Strategy ~. + « «© « «© « «© «© © «© «© @ « 


IV-8 


IV-9 


IV-10 


IV-10 


IV-11 


IV-12 


IV-12 


IV-14 


C. 


APPENDIX. 


2. Organizational Tug-of-War . . « « « « -« 
3. The Pressure of Time « « « « « © « « « « 
4. 111-Prepared Hypotheses . . « « « « « e« 
5- Changes in Context - « « « « « « © « « « 
6. The Uncertainties of Behavioral Sciences 
Strengths. .« « « 6 « « «© © © « © © © © @ © eb 
1. Agency Responses « « « « © « «© « «© « « « 

-~ Food and Drug Administration. .« « « 


-~ Federal Communications Commission . . 


Federal Supply Service. .« « « « « « « 
-- Securities and Exchange Commission . 
-- Office of Technology Assessment .« « -« 
Ze che ETIP Strateg yous cel oot clesmeuie (eure 
-~- Partnership »« « « « « « © « e «© « «© « 
-- Choice of Partners. « « « « « « « « « 
-- Staff-to-Staff Collaboration. ... . 
-- Seed Money. « « « « © « « © © © © «© «@ 
-- Non-Emergency Role. « « « « « « « « « 
-- Transferral of Skills and Ownership . 
-- Adaptability. . « « « « « « © © « © e 
ss— MODES (Vite ledlsule lets l/s! stasmiiclls tl sire ureiis 
-- Development of Commonality. . ... . 
~- Symergic Strategies .« . « « « « © « e 
oop ehh Mr ee ey Bee oe oe eee a ar a 


vi 


EXECUTIVE SUMMARY 
A. HISTORY AND STRATEGY 


From 1972 to 1982 the Experimental Technology Incentives Program (ETIP) 
of the National Bureau of Standards carried out more than 200 experiments and 
studies to better understand how government policies impact private-sector 
inventiveness and to improve those impacts. 

ETIP adopted a strategy of experimentation with incremental changes in 
administrative policies and procedures in Federal agencies. Major components of 
this strategy included: 

o Selection of agencies already inclined toward change where it was 
hypothesized the change, if properly planned and implemented, 
would result in an improved environment for private-sector innova- 
tion; 

o Collaborative problem-solving by involving all stakeholders in 
the planning, implementing, and evaluating of the experiment; 

o ETIP’s role of neutral third-party facilitator; 

o Continuing feedback of data on the experiment’s private-sector 
impacts for use in evaluating results and planning successive 
modifications and incremental changes; and 

o Emphasis on transferring ETIP’s skills, perceptions, and methods 
to line agencies and institutionalizing those changes found bene- 
ficial. 

This strategy meant that ETIP served both as a catalytic agent for change 
and as a student of change and its effects. ETIP applied the strategy to four 
areas of Federal policy and activity: regulation, procurement, economic 
assistance, and resea*ch and development. 


Approximately $2: million was appropriated by the Congress to ETIP during 


its ten years. Begin ing with a Domestic Policy Review of Innovation effort in 


wgakos 


effort in 1979, ETIP activities were redirected to new initiatives and no new 


experiments were permitted. Phasing out of ETIP activities continued with the 


change of administrations in 1981 and the program was terminated at the close of 


1982. 


Be. ACHIEVEMENTS AS CHANGE AGENT 


le Regulatory Program Area 


Following are among the major marketplace and government agency changes 


that resulted from ETIP regulatory experiments and studies: 


O 


A study of economic impact data prepared by experts in the 
field led to deregulation of rail transport of fresh produce 
and of intermodal transports. This resulted in adoption of new 
technologies by the railroad industry. 


Institu:ionalization in the Food and Drug Administration (FDA) 
of a systematic program of post-marketing surveillance for 
adverse drug reactions and other procedure modifications held 
promise that drug-approval time might be reduced. 


The public message services industry was opened to competition 

by changes in policies and procedures of the Federal Communications 
Commission (FCC). An approach for developing and using data- 
collection techniques to monitor the telecommunications industry to 
inform future decision-making by FCC was institutionalized. 


we Procurement Program Area 


Following are among the major marketplace and government agency changes that 


resulted from ETIP procurement experiments and studies: 


oO 


Innovative techniques (Life Cycle Costing, Value Incentive Clauses, 
Performance Specifications) were institutionalized at the Federal 
Supply Service (FSS), resulting in cost-savings and some directly 
attributable product innovations. 


A comprehensive Life Cycle Costing approach to analysis of options 
for meeting Federal space requirements was established for the 
Public Buildings Service (PBS). 
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oO 


A "how to do it manual," training sessions, technical assistance, 
and documentation of success stories were provided for those 
interested in using Life Cycle Costing techniques. 


Innovative procurement strategies were adopted by mission-oriented 
Federal agencies with built-in incentives for fostering technologi- 


cal advance (e.g-, Department of Defense (DOD), Department of Energy 
(DOE), Environmental Protection Agency (EPA)). 


These procurement techniques were formally adopted as part of on- 


going government policies by the Office of Management and Budget 
(OMB) - 


3. Economic Assistance Program Area 


Following are among the major marketplace and government agency changes that 


resulted from ETIP studies and experimentation with economic assistance programs: 


oO 


The Small Business Administration (SBA) increased its intervention 
in regulatory rule-making that affects small businesses and stepped 


up technical assistance to help small firms meet regulatory require- 
ments. 


The Securities and Exchange Commission (SEC) institutionalized a 
computer-based data-gathering, monitoring, and evaluating model to 
inform regulatory decision-making. Resulting rule and reporting-form 
changes substantially reduced the burdens cn small technology-based 
firms. 


Guidelines were developed for future efforts by quasi-public bodies 
to inject capital funds into product development, based on monitoring 
of the Connecticut Product Development Corporation (CPDC). 


as Research and Development Program Area 


The following are among the major marketplace and government agency changes 


that resulted from ETIP studies and experimentation with Federal research and 


development programs: 


O 


Three major institutions committed themselves to construct Integrated 
Utility Systems (IUS) as the result of ETIP-funded feasibility 
studies, demonstrating the value of this strategy over traditional 
Federally-fuided demonstration projects. 


Guidelines were developed for using R&D consortia as a result 


of ETIP’s organization and supervision of a consortium to develop 
fabric flame-retardants. 


il 


Other R&D Program Area initiatives by ETIP were studies of the impacts 


of traditional Federally-funded R&D and demonstration projects which generated a 


number of guidelines for future efforts, discussed in Chapter IV of this report. 


The guidelines were adopted formally by EPA, OMB, and the Energy Research and 


Development Administration (ERDA) as a result of ETIP’s collaborative efforts 


with those agencies. 


C. LESSONS LEARNED 


At the outset almost nothing was known about how Federal actions impacted 


private-sector innovation.e In its ten years, ETIP made important advances in Ns 


filling that knowledge gap. The major lessons learned: 


oO 


The interplay of government and industry is more complex and 
involves more factors than the two-sided equation traditionally 
assumed. To be effective government must tailor policies and 
procedures to the broad spectrum of marketplace forces. 


Considerable time, money, and care need to be invested in experimen- 
tation with incremental government change if government activities 
are to be modified in a credible and effective way to influence 
technological innovation. 


Any strategy aimed at modifying government behavior so as to 
foster private-sector innovation must necessarily involve stake- 
holders from all sectors and should rely on credibility built 

by use of outside experts in the substantive fields. 


Interviews to determine stakeholder expectations from the experimen- 
tal changes should be done at the outset to clarify objectives and 
assure that those objectives are realizable. 


Protection of developers’ proprietary rights in new products 
and product innovations appears essential if government inter- 
vention in the private sector is to have a favorable and signi- 
ficant impact on technological innovation. 


Top-level agency commitment to experimentation is necessary to 
keep political and other pressures off agency experimenters and to 
avoid disruptions in what is necessarily a long-term effort. 


Line-agency staff and decision-makers have differing needs from 
any given experiment with change. All these needs should be 
addressed in the design of the experiment. 


Line agencies need analyses of economic-impact data in order to 
base decision-making on real-world events. Conventional economic 
tools cannot be applied directly to policy analysis. 


Retrospective analyses and evaluations of government activities 
can never substitute for the benefits derived from real-time 
monitoring and feedback of impact-data to decision-makers. 


In addition, ETIP generated a number of program area-specific lessons, 


discussed in Chapter IV. Following is a sampling of the most significant: 


18) 


Government’s regulatory actions significantly affect private 
industry’s willingness to venture into innovative technologies. 
Chief among these impacts is doubt about regulatory intentions. 


A market mechanism approach to regulation is more desirable and 
efficacious than the traditional command/control strategy. 


The most important factor influencing whether procurement and 

other government incentive strategies will affect industrial innova- 
tiveness is the stage of maturity of the product and its industry. 
The impact is likely to be greatest at the earliest stages of 
product/industry development. 


Federal procurement’s influence on innovation can be expected to 
be most effective when government is itself the major purchaser 
and end user of the innovation. Cooperative agreements are the 
preferred strategy of procurement when government is not the 
major purchaser or end usere 


Procurement strategies aimed at stimulating industrial innovation 
frocurement & & 


can be effective when they are congruent with commercial market 
strategies. 


Government assistance can have a direct effect on firms pursuing 
incremental innovation strategies but only indirect effects on 
those pursuing major innovation strategies because of the inter- 
play of key variables in such ventures. 


Subsidies have their greatest impact on technological change 


in those cases where they are a relatively important element 
in a set of coordinated government programs specifically aimed 
at technological innovation and diffusion. 


Policy-makers in subsidy-granting agencies apparently pay little 
attention to the possible impacts of subsidies on technology change 


a 


and hav.: missed opportunities to use technological change to help 
achieve the goals of the subsidy programs. 


Stockpiling is not generally an appropriate government strategy 
against critical material shortfalls. R&D aimed at producing 
substitution technologies in line with projected mineral costs 
might be appropriate. 


Small businesses face special problems in their attempts to meet 
Federal regulatory requirements. 


R&D and demonstration projects have had limited value in fostering 


diffusion of new techologies because each is but a small part of the 
total cost of bringing technological innovation into the marketplace. 
R&D funding policies should be formulated in the larger context of 
the complex process of innovation. 


Technology is more likely to be "pulled" into the marketplace by 
favorable conditions among potential users than it is to be "pushed" 
by government subsidy or R&D funding. 


Before embarking on subsidization, R&D funding, or a program of 


demonstration grants, government should explore alternative strate- 
gies, including feasibility studies and consortium techniques. * 


De CONCLUSIONS 


Throughout its history ETIP learned about government-to-government change 


inducements from both its successes and failures. Its weaknesses derived 


in large measure from the "real-world" environment in which its experimentations 


took place -- an environment which at best provided only moving targets and 


required ETIP to engage in the less-than-exact behavioral sciences. 


Line-agency staff which worked most closely with ETIP lauded its endeavors, 


its skills, and its seed money as essential to the successful changes implemented. 


The strengths derived from ETIP’s strategy included: 


Oo 


The development of partnership relationships with agency "targets 
of opportunity," which contributed to successes and generated ETIP 
crediblity among other agencies; 


Collaborative techniques by which ETIP avoided developing "authority" 
over its partner agencies; 
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oO 


Provisiou of seed money to budget-constricted line agencies which 
otherwise cculd not have funded the experiments; 


Protection from being drawn into the role of "fire-fighter," 
which helped protect ETIP’s commitment to long-term experimenta- 
tion with incremental change; 


Emphasis on the transferability of ETIP’s skills, perceptions, 
and expertise, which resulted in a disproportionate impact by 


being able to reproduce itself within its partner agencies; 


The ability to adapt its approach to the uniqueness of each 
situation; 


The ability to develop common goals, common language, common 
understandings, and common strategies with agency partners; and 


Avoidance of adversarial strategies. 


ETIP successfully showed that monitoring the private-sector impacts of 


government activities and modifying those activities based on analyses of the 


results of that monitoring can and will foster technological advance in the 


private sector. Further, it showed that such monitoring and analyses for 


appropriateness can be institutionalized in government agencies to support 


incremental change in policies and procedures. Finally, ETIP developed consid- 


erable knowledge about government-industry interactions that can serve future 


decision-making, should the need for experimentation with government activities 


to encourage technological innovation be rediscovered. 
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I. INTRODUCTION 


The President’s 1972 Message to the Congress on Science and Technology’ 
stated: 


The ability of the American people to harness the discoveries of 
science in the service of man..-has undergirded both our domestic 
prosperity and our international strength... (pe 1). 


-ee(T)he mere act of scientific discovery alone is not enough. 

Even the most important breakthrough will have little impact on our 
lives unless it is put to use -- and putting an idea to use is a far 
more complex process than has often been appreciated... (p. 2). 


---(W)e must see that the environment for technological innova- 
tion is a favorable one- In some cases, excessive regulation, 
inadequate incentives, and other barriers to innovation have worked 
to discourage and even to impede the entrepreneurial spirit. We 
need to do a better job of determining the extent to which such 
conditions exist, their underlying causes, and the best ways of 
dealing with them... (p- 2). 


There are many ways in which the Federal government influences the 
level and the quality of private research and development. Its 
direct supportive efforts are important, but other policies -- such 
as tax, patent, procurement, regulation, and antitrust policies -- 
also can have a significant effect on the climate for innovation... 
(p- 6)- 


---(W)e need to be better informed about the full consequences of 
all such policies for scientific and technological progress (p- 7)- 


The Experimental Technology Incentives Program (ETIP) was initiated in FY 
1973 in the National Bureau of Standards as a pioneering effort to understand 
and improve the three-way relationship of government, free enterprise, and 
technological advance. Its express purpose, as stated by the President, was 
"to determine effective ways of stimulating non-Federal investment in research 
and development and of improving the application of research and development 


results" (p. 8). The program was terminated almost a decade later, at the end 


of 1982. 


1. "Message from the President of the United States Concerning Science 
and Technology," 197’ (House Document No. 92-193). 


Early in its evolution ETIP focused its efforts on government, primarily 
Federal agencies. Their accessibility and, in many cases, willingness to 
change made them ideal environments in which to experiment with techniques to 
foster private-sector technological innovation. During its ten-year lifespan 
more than 200 projects explored how government policies and procedures affect 
the pace, direction, and deployment of technological innovation in the private 
economye The research, in turn, was to suggest changes in government that would 
remove obstacles to innovation and replace them, where possible, with incentives. 
ETIP was the subject of repeated evaluations. Gerald M. Britan, Assist- 
ant Professor of Anthropology and Urban Affairs at Northwestern University, 
conducted one of these for the National Academy of Sciences between 1975 aa 
1977. He wrote:! 
eee(By 1975) i was already clear that ETIP was itself a far more 
significant experiment than any of its policy initiatives. The 
program represented a radical innovation in the process of policy 
change. 
This, the final ETIP report, is intended to be an overall record of the 
ETIP experience: the course of its development, the nature of its experiments 
in government change, its accomplishments in altering both government percep- 
tions and practices and their resultant impacts on private-sector innovation, 
and the lessons ETIP learned about the interplay between government policies 
and private initiatives. As such it serves as a road map for those who wish to 
study in detail the history of ETIP’s specific activities. The primary audience 


for the report consists of students and practitioners of public administration 


1. "Bureaucracy and Innovation, An Ethnography of Policy Change," Gerald M. 


Britan, Sage Library of Social Research, Vol. 115, Sage Publications, Inc.-, 1981, 
De OT 
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and policy-making who are confronted with questions of how better to organize 
government activities and improve their impact on the industrial economy. 

The report is not intended to repeat the detailed records and data on each 
of ETIP’s studies, experiments, and other activities. Those records are avail- 
able from other sources. Interested parties can locate them using footnotes in 
this report and the Appendix. 

The report has been prepared by persons not directly involved in ETIP in 
an attempt to avoid bias. It does not contain new information or analyses. 
Rather it is a distillation of information gathered from interviews with ETIP 
personnel and from internal ETIP documents, project reports by ETIP and its 
several outside contractcrs, and evaluations of its activities by numerous agen- 
cies, including the Department of Commerce, other Federal agencies, review 
panels established under the auspices of the National Research Council, and 
outside contractors. 

Chapter III highlights a variety of ETIP projects, their task objectives 
and their results. Those projects are ineduded which best illustrate the ETIP 
method and its application and achievements. Chapter IV discusses those same 
projects and others in terms of the "lessons" ETIP learned about how government 
can influence private-sector innovation. 

Task objectives for each ETIP project frequently were modified during the 
life of the particular project as the result of mid-course findings. Thus, 
simple comparison of results with the initially stated objectives of each pro-— 
ject is not always useful or instructive. However, ETIP did establish these 
three overall objectives for its program: 


o To develop generalizable information on how government processes 
impact private-sector innovation; 
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To test hypotheses on how government processes can benefit private- 
sector innovation; and 


To construct an experimental methodology for agencies to use in 
reforming their processes of implementing policy. 


Because of the diversity of points of view among those who participated 
in ETIP activities and the difficulties posed in measuring results against 
stated objectives, this report does not attempt to state "official" conclu- 
sions. Rather, Chapter V presents the impressions of a variety of persons who 
were involved with ETIP either as participants or as close observers, evaluators, 


and/or users of its experiments and their results. 


if 
II. HISTORY AND STRATEGY 


Ae BACKGROUND 


In 1971 several studies were commissioned under the New Technology 
Opportunities Program of the Domestic Council to determine how science and tech- 
nology initiatives could be used by the Federal government to spur the nation’s 
economy. 

These studies resulted in many proposals for Federal research and develop- 
ment (R&D) projects, but also showed that surprisingly little was known about 
how such federally-initiated R&D actually affects technological innovation in the 
marketplace. It was found, for example, that such factors as customs, regula- 
tions, Federal procurement policies, tax laws, subsidies, and general financial 
conditions also affect the entrance of R&D-spawned technology into the private 
sector, but the lack of information about the interplay of R&D and these other 
factors made it difficult to select projects and programs which would best affect 
the pace and direction of private-sector innovation. 

To supply the needed information, the President in 1972 directed the 
National Bureau of Standards to establish the Experimental Technology Incentives 

1. This section is drawn from several documents devoted to ETIP history: 
"Experimental Technology Incentives Program, History Development," 1976, a 
briefing paper written by ETIP for the Department of Commerce; ''Potential 
Problems That Should Be Considered in Evaluating the Experimental Technology 
Incentives Program," 1977, Government Accounting Office overview of the program; 
"Public Policy and Innovation: An Overview of the Experimental TechnologyIncen- 
tives Program," 1977, Gerald M. Britan, an evaluation of ETIP for the National 
Academy of Sciences; "The Experimental Technology Incentives Program: Progress 
and Impediments, An Information Base for Program Evaluation," 1977, Thomas V. 
Kelly, Department of Commerce evaluation; and "The Experimental Technology 


Incentives Program, Brief History and Major Accomplishments," 1981, a briefing 
paper written by ETIP for the Department of Commerce. 
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Program (ETIP) "to determine effective ways of stimulating non-Federal investment 
in research and development and of improving the application of research and 


nl The National Science Foundation was directed to 


development results. 
establish a supporting program to generate ideas and opportunities for ETIP 


experiments. 


Be EXPERIMENTATION STRATEGY 


The strategy adopted by ETIP was to develop experiments jointly with 
selected government agencies in which, it was hypothesized, agency policies and 
tools might be modified to spur the rate at which technological innovations were 
developed and introduced into the private sector. Such experimentation itself 
was a novelty in government, where traditional strategies had been R&D demonstra- 
tion projects and "blind leap" policy modifications which may or may not have 
proven useful or beneficial. As pointed out in an article early in ETIP’s 
history, "In an experimental Baer ye Pa high rate of failure can be 
condoned and viewed as a learning experience. No is a valid sadweehl 

The strategy involved actual policy-makers in the ETIP process, offered 
ETIP opportunities to study a broad range of policies and tools believed to 
affect innovation, and provided a manageable way of obtaining information pre- 
viously lacking. 


The approach came to be identified within ETIP as "administrative experi- 


mentation" -- introduction of change in a systematic way that advanced only a few 


ibe Op. Cites pe I-l, p-8. 
2. "“ETIP: A Remarkably Better Way," by John F. Judge, Editor, Government 
Executive, July 1975. 
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steps at first, collection of data about initial effects of the changes, and 
information feedback for use in planning incremental steps until the new policy 
or process was institutionalized or rejected. 

By design, administrative experiments had twin objectives. One, that of the 
participating agency partner, was to find ways to improve agency performance and 
effectiveness. The other, that of ETIP, the researcher, was to identify how 
changes in agency activities would affect innovation. Thus, for an experiment to 
take place the critical factor needed was a mutual interest in change. Agencies 
offered the policy-making environment and an interest in changing their operations 
for their own reasons, which sometimes included an interest in technological 
innovation. ETIP offered resources for designing and evaluating the experimental 
changes.- Each partner benefited, although in different ways. In practice, ETIP 
became both a student/researcher of the change process and a catalyst/facilitator 
of that change. Program results became closely identified with the goals of 
specific agencies with which ETIP worked, and success of the experiments was 
defined as the development and use of hentseie and innovation-stimulating 
processes by those partner agencies. 

Administrative experimentation, as developed, funded, and implemented by 
ETIP, had two major advantages. First, it allowed ETIP and its stakeholder- 
partners from both government and industry to gain actual experience with 
policy change. Second, it afforded both line-agency decision-makers (who would 
be expected to approve the change found valid in the experiment) and line-agency 
staff (who would be expected to implement it) an opportunity for firsthand 
experience with the change, which proved invaluable in facilitating the transi- 


tion from experiment to formal implementation. 
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The key components of the strategy included: 


o Collaborative problem-solving. ETIP actively sought throughout 
an experiment, from identification of initial hypotheses to be 


tested through the design, implementation, mid-course correction, 
and final evaluation of each project, the participation of all 
involved stakeholders, including government line-agency personnel, 
relevant industry and user groups, and academic and legislative 
interests. It found this a particularly safe way to bring public 
and private sector groups together as partners rather than in 
their more traditional roles as adversaries. 


o Third-party facilitation. ETIP adopted in each experiment a neu- 
tral third-party role to effect this collaboration. As facilitator 
it provided seed money that frequently opened doors to agencies 4nd 
fostered their willingness to experiment with change they could not 
otherwise have done because of funding limitations and the consequent 
need to divert funds from ongoing, mission-oriented activities. 
Further, both government mission-oriented agencies and the private- 
sector groups with which they dealt perceived ETIP as having "no ax 
torgrind.. 


o Focus on expectations. At the outset of each experiment ETIP inter- 
viewed all stakeholders to learn the expectations they brought to 
the experiment. Arraying these expectations helped clarify the 
objectives being sought and determined whether those objectives were 
in fact realizable. 


o Feedback of field data and evaluability. Ongoing collection of 
field data during experiments.was essential to the ETIP process. 


It afforded continuing involvement by all stakeholders, provided 
experimenters with information needed to make mid-course correc- 
tions in the experiment itself, and allowed ongoing evaluation of 
experiment results. Furthermore, it set in motion the apparatus and 
procedures for continued gathering of field data needed by line- 
agency decision-makers for further incremental changes in policies 
and procedures and for instituting new activities or abandoning old 
ones found deleterious. 


o Emphasis on transferability. The end result sought by ETIP was 
institutionalization of those concepts and policies proved valid 


by an experiment and, even more important, institutionalization 
within line agencies of the ETIP process itself -- the willingness 
to experiment with incremental change in policy and procedures 

in a way that would result in valid, credible answers. 


ETIP intervention in government procedures itself represented an innovative 


blend of "science" ani "politics." The administrative experiment was neither 
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"pure science" subjected to the rigors of controlled laboratory or field exper- 
imentation methodology, nor "pure politics" freed of all scientific constraints 
for data analyses and evaluation of results. 

Evaluability became a keystone of the ETIP method as a result of mid- 
course corrections made in 1977. ETIP’s evaluability assessment contractor was 
an essential participant and a stakeholder in each of ETIP’s experiments. Eval- 
uation of results was designed into each project at the outset, and monitoring 
for evaluability was made an integral and ongoing part of each project. 

Ongoing evaluability engendered both a flexibility in project design that 
permitted "fine tuning" during the course of the project and the "scientific 
rigor" of assuring that project results could be subjected to analysis which 
would yield valid responses to hypotheses being tested. Additionally, the stra- 
tegy helped ETIP earn credibility among line agencies and other stakeholders as 
a neutral third-party "facilitator" interested in those stakeholders’ objectives 
(e.g-, improved line-agency operations, improved industry profitability) as well 
as in ETIP’s own objective of deriving wei ua information about government- 
industry interactions. 

The evaluability contractor, the Performance Development Institute (PDI) 
of Washington, D.C., noted both special assets and limitations in the overall 
administrative experimentation strategy: | 

The interactive process of administrative experimentation provides a 
useful basis for making modifications as events unfold. However, the 


nature of this approach makes it unlikely that broad generalizations 
will be developed as quickly. Special synthesis work is necessary..-- 


1. "Finding and Encouraging Innovation: Approaches to Investigating the 
Relationships Between Industrial Innovation and Regulation," Warren Frederick, 
Joe Nay, Performance Development Institute, Washington, D.C., June 1980, pp. 
52-53. 


Only to the extent that there is a very specific administrative purpose 
involved, will this approach suggest "what should be".... 


C. PROGRAM AREAS AND PROJECTS 


Although the President’s 1972 message focused exclusively on Federal R&D, 
ETIP recognized early that a much wider range of government activities would 
have to be considered. In fact, by its termination there was a growing consen- 
sus within ETIP that innovation was influenced by a broad spectrum of government 
agencies and programs that at first glance have little or no bearing on new- 
product research and development. 

ETIP examined a wide range of factors and gove -nmental activities believed 
to affect the willingness of the marketplace to develop and adopt new technology. 
Four program areas were targeted for experimentation in 1974: procurement pol- 
icy, regulatory policy, private-sector research and development policy, and 
small-business policy. Advanced planning and research was subsequently added 
to ETIP activities to provide overall project development, planning, and evalua- 
tion. In 1975, the small business program area was expanded into a new economic 
assistance program area and an experimental methods area was created to design 
and evaluate projects. 

A partial list of ETIP activities (some of which are discussed more fully 
in Chapters III and IV) illustrates the range of agencies with which ETIP devel- 
oped a working partnership and the breadth of projects undertaken with these 


partner agencies. E' [IP worked with: 
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Government Regulation 


oO 


Procurement 


oO 


The Federal Communications Commission (FCC) in deregulating and 
fostering competition in the public record message industry 
(Project Nos. 140 and 161); 


The Federal Railroad Administration (FRA) and the Interstate Com 
merce Commission (ICC) in deregulating rail shipments of produce 
(Project No. 31); 


The Environmental Protection Agency (EPA) in reforming reporting 
requirements for pesticide firms to eliminate unnecessary burdens 
on industry (Project No. 48); 


State public utility commissions in developing ways to reduce the 
time required to decide rate cases and in introducing analyses for 
rate-case decision-making, thus facilitating introduction of 
cost-cutting technologies (Project No. 76); 


The Food and Drug Administration (FDA) in improving the drug-approv- 
al process by developing ways to anticipate regulatory burdens on 
bringing new drugs to market and by developing cost-effective post- 
marketing surveillance programs (Project No. 88); 


The Federal Supply Service (FSS) in a series of experiments apply- 
ing Life Cycle Costing, performance specifications, and Value 
Incentive Clauses to government procurement (Project Nos. 2, 36, 
63, and 64); . 


The General Services Administration (GSA) in developing methods 

for using government procurement (a) as an early market for innova- 
tions, (b) to develop prototypes, and (c) in market-testing of new 
innovations; 


EPA, the Departments of Transportation (DOT), Defense (DOD), and 
Energy (DOE), and the Office of Management and Budget (OMB) to 
institutionalize innovative procurement strategies designed to fos- 
ter technological advance; 


Economic Assistance 


The Securities and Exchange Commission (SEC) in developing and 
institutionalizing a computerized model for evaluating the private- 
sector economic impacts of changes in rules and regulations 
(Project No. 148); 


O 
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The Small Business Administration (SBA) in redirecting its activi- 
ties to help small technology-based firms meet regulatory require-— 
ments (Project No.- 28); 


Outside consultants in evaluating a range of Federal policy tools 
(e-g-, subsidies, R&D funding, tax and trade laws, commodity stock- 
piling) which would help the private sector in its innovative 
endeavors while helping the Federal government achieve national 
policy goals (Project Nos. 15, 17, 93, 130A, 132, and 167); 


The Connecticut Produce Development Corporation (CPDC) in evaluat-— 
ing its effectiveness in fostering development of new products by 
small high-technology firms and developing guidelines for future 
similar efforts (Project Nos. 32 and 78); 


Research and Development 


oO 


The Department of Health, Education and Welfare (HEW) in demon- 
strating the feasibility of commercial application of Integrated 
Utility Systems (Project No. 67); 


A Clemson University-led consortium for developing flame-retar- 
dants for apparel fabrics in a demonstration of the merits of the 
consortium approach to R&D (Project Nos. 7, 97, 125, 126, and 134); 
and 


EPA, OMB, and the Energy Research and Development Administration 
(ERDA) in incorporating within those agencies” procedures and pol- 
icies a set of guidelines for structuring and/or approving demon- 
stration project budgets. 


ETIP received nine appropriations from the Congress totaling approximately 


S25 million. 
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Distribution of these funds among ETIP activities was as follows: 


Staff and administration -- $6.5 million; 
Procurement projects -- $6.5 million; 
Regulatory projects -- $5 million; and 


Economic assistance, civilian R&D, and small business projects -- 
$5.5 million. 


The remaining $1.5 million was used in support of a Domestic Policy Review 


of Innovation (DPR) and related tasks within the Department of Commerce. The 


DPR was established in 1979 to determine what had been learned about stimulating 


innovation in the private sector and what new initiatives should be undertaken. 


It was similar in nature and scope to the previous effort which had resulted 


in the establishment of ETIP. 


D.- PROGRAM CLOSURE 


During the DPR the reorganized Office of the Assistant Secretary of 
Commerce for Productivity, Innovation, and Technology concluded that ETIP had 
demonstrated a number of ways in which government could stimulate technology 
and that these mechanisms should be passed to the appropriate agencies. In ef- 
fect, this was a determination that ETIP was no longer "experimental" and that 
its work was completed. The Assistant Secretary redirected the program towards 
support of new initiatives recommended by the DPR: studies of patent policy, a 
program for utilizing technology developed in Federal laboratories, studies of 
the feasibility of developing reference services to provide industry access to 
Federal technological information, and development of models to assess indus— 
trial sectors where government ThPervenrd On might be warranted to stimulate 
technology. Remaining ETIP projects were to continue to completion, but no new 
experiments were to be initiated. 

ETIP was renamed the Office of Strategy and Evaluation and was to be moved 
into the office of the Assistant Secretary as support staff in 1980. The change 
of administrations in 1981 interrupted this reorganization but continued the 
phasing-out of ETIP as ordered by the departing Assistant Secretary. Experi- 
ments with economic assistance and procurement were completed in 1981, and those 
with regulatory policies were terminated in 1982. Technology studies begun as 


DPR initiatives were completed in 1981. With no new funding from the Congress, 
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the Experimental Technology Incentives Program ended in late 1982. Its final 
activities were to transfer to interested agencies the knowledge it had gained 
and to leave a record of its activities and accomplishments for those interested 
in facilitating innovation in government and industry. 

This final report, summarizing as it does the ETIP experience, its suc- 
cesses and failures, and the lessons that were generated by both, can be con- 
sidered a final installment of ETIP information considered useful to agencies 


interested in profiting from ETIP by institutionalizing its results and methods. 


III. ACHIEVEMENTS -- ETIP AS CHANGE AGENT 


The achievements of ETIP can be assessed from each of three vantage points. 

1. The Marketplace: Did ETIP’s experiments succeed in stimulating pri- 
vate technological initiatives or, at the least, in removing govern- 
ment obstacles to them so as to permit those initiatives to take place 


and new products to be introduced into the marketplace more efficient- 
Lyi 


2. Government Line Agencies: Did ETIP succeed in redirecting or other- 
wise altering government agency policies or procedures in the belief 
that such changes would result in an improved climate for technologi- 
cal advance in the private sector, even if new products were not 
immediately developed and/or introduced into the market- place? 


3. ETIP: To what extent did the program achieve its overall objective, 
that of developing an understanding of: 


o The complex process by which technology is introduced into 
the marketplace; 


o The government impediments to such introduction and bow their 
effects might be alleviated; 


o How government policies and procedures might be used to stimu- 
late innovation; and 


o The most effective ways af bringing about government innova- 
tions that can in turn stimulate the desired technological 
advance. 

This chapter addresses the first two of these measurements. That is, it 
discusses ETIP’s achievements as change agent within government end the effects 
of those government changes on technological inncvation in the private sector of 
the economy. Although considerations from each of the three vantage points 
necessarily overlap those from the other two, Chapter IV will attempt to isolate 
and discuss ETIP’s achievements as a student of the innovation process: 
the lessons it learned about the complex interweave of goverament and private 


sector actions that results in either successful and efficacious diffusion of 


new processes and tecnnology or the hindrance of such diffusion. 
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As previously described, ETIP experiments and studies were conducted in 
each of four areas of government activity: regulation, procurement, economic 
assistance, and research and development. Accordingly, this chapter organizes 


ETIP activities under these same headings. 


Ae REGULATION 


The Federal government’s regulatory power is exercised for a variety of 
reasons, among them protection of public health and safety, protection of the 
environment, and avoidance of undesirable commercial practicese Inasmuch as 
regulations normally are written to achieve a public protection objective, the 
effects of such regulations on technological advance generally have not been 
considered by the regulating agencies. Thus, it was hypothesized by ETIP, some 
regulations act as unnecessary barriers to innovation by increasing market un- 
certainties, by raising the costs of new product development, and by distort- 
ing investment allocations in the "free were These, it was posited, could 
become increasingly deleterious unless adequate consideration was given to en- 
sure that a favorable climate for technological advance prevailed. 

The primary concern of ETIP’s regulatory area activity was "to develop 
knowledge about changes in regulatory processes that might provide incentives 
or reduce barriers to technological innovation." 

Under this umbrella objective ETIP sought to establish the credibility of 
administrative experimentation as a policy improvement tool within regulatory 


1. “Experimental Technological Incentives Program Regulatory Policy Area 


Plan," Daniel W. Fulmer, Regulatory Group Leader, Experimental Technology Incen- 
tives Program, June 28, 1977, p. l. 
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agencies and to develop and refine a strategy for experiment selection and 
management. 

ETIP’s "administrative experimentation" approach to regulatory reform was 
based on recognition that regulatory agencies historically had been faced with 
only two alternatives -- the status quo or "blind leaps into an uncertain 
future."+ The "administrative experiment" offered a third option: introduc- 
tion of incremental change in a planned, systematic, and data-supported way. 
Much of the work was performed in the field and with as much teamwork as possi- 
ble among parties with differing viewpoints. Additionally, using interviews to 
gather information encouraged development of consensus where it might not other- 
wise have occurred throush formal inquiry or written docket procedures. 

Using this approach, ETIP developed partnerships with regulatory agencies 
which wished to effect s gnificant operational change and were also responsive 
to ETIP’s experimentation goals- Conversely, ETIP was viewed as responsive to 
the administrative goals of the partner agencies. 

The regulatory experiments that resulted involved ETIP in activities with 
a number of Federal agencies including, among others, those discussed in this 
section: the Federal Railroad Administration (FRA) and Interstate Commerce 
Commission (ICC), the Federal Communications Commission (FCC), and the Food and 


Drug Administration (FDA). 


l. FRA/ICC -- Fresh Produce Rail Transport Deregulation (Project No- 31) 
In the 1970s the railroad industry had become increasingly alarmed that a 
rapidly declining portio of perishable produce was being shipped by rail. It 
1. Preface of "Toward Competitive Provision of Public Record Message 
Services?" by Daniel W. Fulmer, Regulatory Group Leader, Experimental Technology 


Incentives Program; Interim Report by The Regulatory Center at the Performance 
Development Institute, Washington, D.C., October 1981 (NBS-GCR-ETIP 81-97) p- iv- 
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was argued that ICC rate-setting placed railroads at a competitive disadvantage 
with the trucking industry, whose produce shipments were not subject to ICC 
rate-setting, and thus made rail shipment of fresh produce too unprofitable to 
permit investment in new technologies, some of which were already available or 
ready for development. 

ETIP involvement began in 1974 with a study funded jointly with the 
Federal Railroad Administration (FRA) and conducted by Manalytics, Inc-, of San 
Francisco, California, and Reebie Associates of Greenwich, Connecticut. The 
study team analyzed data on sample shipments for selected commodities from four 
areas to five distribution markets and conducted interviews with approximately 
500 stakeholders. The study resulted in an 1l-volume report! which concluded 
that rail shipment of fresh produce would disappear in the near future unless 
the industry invested in new intermodal equipment. 

e+ e(T)he trucking industry will remain the "backbone" of produce 
transportation. However, there is a "viable role" for rail transpor- 
tation in this market, if the problems...(of) rail service, rates, 
profitability, and equipment are- addressed. A viable role can be best 
achieved if railroads, shippers, and receivers make necessary service, 
financial, and volume commitments. 

-ee(T)he most important single step...is exemption from Commission 
rate and service regulation. This is a necessary prerequisite for 
the negotiation of rate and service contracts "outside the normal 
regulatory environment, with ifs procedures for protest and commis-— 
sion review" (Vol. 2, p- 28). 

ETIP contractors briefed ICC policy-planning staff on the findings. As a 
result of that briefing and a series of exploratory meetings, ETIP and ICC 
entered into an interagency agreement to conduct regulatory administrative 

1. "A Long Term Study of Produce Transportation," Federal Railroad 


Administration Report, December 1977 (FRA-OOPO-78-2) . 


2. "Interim Report on Potential ICC/ETIP Administrative Experiments," 
March 1980 (NBS-GCR-ETIP 80-85) p- II-10, ll. 
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experiments. Involvement by ETIP, The Urban Institute, and later the Perfor- 
mance Development Institute, contracted to perform project evaluation, was 
governed by two tenets: 

o Any regulatory policy change under consideration should be of such 
a nature and so structured to serve as a sufficient incentive that 
the regulatees (railroads, shippers, and receivers) will respond 
positively by planning for or implementing desirable changes in 
operations and service and equipment innovation. That is, the pol- 
icy change will not be ignored by the private sector. 

o Any regulatory policy change should be capable of being measured 
in terms of the initial response to the policy change (Do regula- 
tees take advantage of regulatory exemptions? Do they make plans 
for changing operations on the basis of changed policy?) and the 
eventual impact and causal linkages to changes, in productivity and 
the introduction of technological innovations. 

In 1978 the Southern Pacific Railroad, using data from the Manalytics/ 
Reebie study, petitioned the ICC for exemption from economic regulation of the 
transport of livestock, fish and shellfish, and unmanufactured agricultural com- 
modities on the grounds that the trucking industry’s transportation of these 
commodities was already exempt. Three months later ICC dismissed the petition 
and initiated Ex Parte 346 rule-making proceedings to consider the issues 
raised by Southern Pacific. 

The Notice of Proposed Exemption explicitly referenced the report prepared 
for ETIP and "the Commission’s public discussion of the proposed deregulation 
indicated substantial reliance upon the factual and analytical basis contained 
in the Manalytics study to support such a proposed policy change. The Commis-— 
sion’s discussion also expressed interest...in treating regulatory policy 
changes as experiments, the consequences of which should be evaluated in order 
to inform decision-making on further deregulation steps.''¢ 


HeeULbade f peer F9PSr4: 
Priiides spon 7. 
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The effort culminated in deregulation of rail produce shipments in March 
1980. An initial evaluation was conducted by ICC at that time, and Manalytics 
and ETIP developed a seminar and report on innovation in rail management that 
immediately followed deregulation. An intermediate evaluation was subsequently 
funded by FRA and ETIP with the cooperation of ICC but budget cutbacks associ- 
ated with the closure of ETIP terminated the project before contract services 
could be procured for the evaluatione ICC did, however, expand the experiment by 
deregulating rail shipment of other commodities and of "piggyback" shipments in 
general. 

These deregulations stimulated new investment in improved technology, in 
cluding "innovations in intermodal railcar design, many of them aimed at weight 
reduction and aerodynamic improvenienten. Further, ETIP succeeded in insti- 
tutionalizing within ICC the concepts that analyses of the economic impacts on 
new technology are proper considerations in rate-setting and other regulatory 
procedures, and that ongoing evaluations of the impacts of regulatory changes on 
technological innovation are legitimate ane informative processes for regulatory 


agencies to institute. 


25 FDA -- Post-Marketing Surveillance of Drugs (Project No. 88)" 


In 1975 ETIP began exploring with the FDA the possibility of speeding and 
otherwise improving the approval process for new prescription drugs. This was 


to be accomplished by an administrative experiment to design, implement, and 


1. "It’s A Growth Market Again," Gus Welty, Railway Age, November 30, 
1981. 

2. This section is drawn in part from "The Experimental Technology 
Incentives Program: Progress and Impediments, An Information Base for Program 
Evaluation," Thomas V. Kelly, Office of Budget and Program Evaluation, U.S. 
Department of Commerce, August 1977, pp. 143-152. 
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evaluate the effects of several components of a comprehensive system for post- 
marketing surveillance of prescription drugs. It was hypothesized that new data 
acquisition and analysis procedures would speed the process without adversely 
affecting public safety, and that this speed-up and consequent reduction in the 
"lag" between new-drug development and commercialization would improve the cli- 
mate for innovation by the pharmaceuticals industry. 

Protracted negotiations preceded development of an ETIP-FDA agreement. A 
year after negotiations commenced ETIP and FDA approved the plan and signed an 
agreement transferring $1.1 million from ETIP to FDA for partial support of the 
project. 

It took two years to develop the experiment from the start of formal con- 
versations through issuance of a Request for Proposal (RFP) in July 1977. To 
some extent this was the result of a conscious ETIP strategy based on prior 
experience --— that a long lead time would enable its staff (a) to become 
thoroughly familiar with the FDA, its context, problems, and potential areas of 
concern; (b) to develop a base of support ts and a working relationship with 
relevant FDA officials; and (c) to "touch base" with members of the academic 
community familiar with FDA. The process of obtaining the interagency agreement 
was also slowed by difficulties in reaching a consensus between ETIP and FDA 
about the nature and purpose of the proposed project. 

A major communication problem developed between ETIP and FDA representa- 
tives. In retrospect, both ETIP and FDA viewed this language problem as part of 
a lengthy mutual education process. FDA representatives had to familiarize ETIP 
staff with the FDA way of doing things while ETIP representatives sought to 


ensure that explicit evaluation activities would be incorporated into both the 
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RFP process and the conduct of the experiment, and that mechanisms would be 
built into the RFP to foster continuing close interaction between FDA, ETIP, 
and the Rone TEOEOr throughout the course of the experiment. This mutual educa- 
tion was viewed by both ETIP and FDA as useful in defining their respective 
areas of concern, although some FDA officials expressed reluctance to endure 
another such language "hassle" as the price of ETIP experimentation. 

At about the time that the substance of the proposed RFP was being approved 
by the FDA’s Bureau of Drugs, the congressionally formed Joint Commission on | 
Prescription Drug Use (JCPDU) became involved. One of the Commission’s objec- 
tives was to encourage post-marketing surveillance for effectiveness (as opposed 
to ETIP/FDA’s objective of monitoring for adverse effects). After initial FDA 
reluctance, it was agreed that representatives from the Commission would ‘be 
included in the Project Advisory Group and that Commission concerns would be 
incorporated in management of the contract. 

IMS America was awarded the contract to develop a prototype or "turnkey" 
method of post~marketing surveillance. In 1978 it became apparent no single 
methcd would work, and IMS and its consultants urged that several components 
would be necessary. 

Meanwhile, pressure developed within the Commerce Department in 1980 to 
reduce ETILP funding of the FDA project because the amount budgeted but as yet 
unspent did not appear to critics of the project to match its prospects for 
reducing the regulatory lag imposed on the drug industry or for otherwise 


improving the climate for pharmaceutical innovations. 
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FDA’s traditional drug approval process was carried out in three phases, 
involving gradually increasing sophistication of the adverse drug reaction in- 
formation sought, gradually increasing numbers of patients being monitored, and 
gradually increasing spans of time. Post-marketing surveillance was carried 
out in some instances in a fourth phase. ETIP sought in its intervention to 
speed this process by using improved data-gathering techniques, by encouraging 
FDA to turn its existing practices into a "system" that would include critical 
self-evaluation, and by encouraging greater government~industry collaboration, 
including a post-Phase II safety review process to determine whether marketing 
of a new drug could commence prior to the beginning of the Phase III trials. 

The late phase of drug investigation (Phase III) is the one in which the 
more difficult to find and more sporadic and unexpected adverse effects are 
sought. Because it was not clearly known how effective such Phase III trials 
were in finding these adverse effects or what precisely was learned from them 
beyond what was discovered in Phase II trials, a retrospective study of Phase 
III trials was started to determine what changes could be made in the drug- 
approval process-e In addition, an industry-government safety review con- 
ference at the end of Phase II trials was added to the experiment. This con- 
ference was to bring together all information already gathered on adverse drug 
reactions and determine whether some types of information could better be 
gathered after marketing and which information could best be learned from Phase 
III trials. This addition to the experiment might lead to changes in the drug 
approval process which would result in earlier marketing, time-saving to indus- 


try, and greater drug~-review effectiveness for FDA. 
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Retrospective reviews of the Phase III and Phase IV studies were planned. 
These and the safety review procedures at the end of Phase II trials promised 
to facilitate joint industry-FDA planning and to further reduce the "lag" and 
improve the climate for pharmaceutical industry innovations. These efforts were 
still in progress in mid-1982 when ETIP was closing down operations. FDA 
decided that post-marketing surveillance experimentation would continue under 
its own sponsorship. ETIP urged FDA also to continue funding on-site evaluation 
and outside-party facilitation of collaboration among industrial, governmental, 
and academic interests. 

The potential linkages between post-marketing surveillance and technologi- 
cal change were still not fully tested by 1982. Still, ETIP could claim success 
for the project in that it sensitized FDA to both the process of administrative 
experimentation and to awareness of (and possible concern over) the ultimate 
technological impact of regulatory policies. Although at first there was little 
substantive concern evidenced at FDA for the impacts their regulations might 
have on technological innovation, there did seem to be a positive feeling for 
administrative experimentation among FDA staff and acknowledgement that without 
ETIP the issue of post-marketing surveillance could not have been addressed in 
such a manner. 

More specifically, ETIP considered the project successful because it 
showed that much more was required in any attempt to improve pharmaceutical 
industry innovation than merely reducing drug-approval time. More important, 
ETIP determined, were the development of collaboration and communication on 


data bases among the parties involved and agreement on systems management, 
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both of which were ETIP=spurred ongoing efforts at FDA at the time of ETIP’s 
termination. 
ae FCC -- Public Record Message Service Deregulation (Project Nos. 140 and 

161 

ETIP first discussed ideas for communications deregulation in 1974. In 
1976 the idea was revived and ETIP contracted TRANSCOM, a small firm specializ- 
ing in communications regulation, to propose ideas for experimentation and to 
provide the expertise that would help "open doors" at FCC. 

ETIP proposed (and FCC accepted in the project plan approved in August 
1977) that the two agencies agree to experiment, allowing the specific nature 
of experiments to be worked out as opportunities and events warranted. Both 
moves, the contract with TRANSCOM and the program plan, represented departures 
for ETIP, which had heretofore selected specific experimentation candidates and 
hired contractors to flesh out experiment designs before signing interagency 
agreements. In this case ETIP relied on outside experts to suggest a range of 
ideas prior to the agreement to ee conta 

FCC’s response to ETIP’s suggestions was favorable. Indeed, FCC staff 
proposed deregulations even more far-reaching than those proposed by ETIP- 
TRANSCOM, including one to open to full competition the public record message 
services (PMS) market, in which Western Union had enjoyed a monopoly since 
1943. 

In 1979 the FCC adopted broader entry and pricing regulations for PMS 
and resolved to monitor the deregulation’s impact on the industry. FCC staff, 


ETIP, and its evaluability contractor, Performance Development Institute (PDI), 
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established a "stakeholders" group to develop a program for evaluating the 
changes and their impacts on the private sectore An evaluation design was 
agreed upon by the group in October 1980, and the following year the evaluation 
of the project was published by ETIP: '"Toward Competitive Provision of Public 


Record Message Services?" The report qerstinrne- 


o While a few firms applied to the FCC and were approved to publicly 
offer record message telecommunications services on a nonsubscrip- 
tion basis, these services predominantly were oriented toward use 
by businesses, not the individual, occasional user. 


o Additional entry to the PMS market was a possibility based on the 
planned activity or past statements of several firms. Furthermore, 
the proliferation of subscription record message telecommunica- 
tions services provided a potential basis for wider public service 
offerings in the future. 


o While development in PMS was slow-paced, business-oriented sub- 
scription record and data telecommunications and teleprocessing 
services became a kaleidoscope of activity. Even greater poten- 
tial existed for new business and residential telecommunications-— 
based services. 


o The development and extent of public availability and affordabili- 
ty of such new services were expected to be substantially influ- 
enced by the eventual outcome: of debate over the proper structure 
of the telecommunications industry and the proper role of govern- 
ment in its regulation. 


o The uneven development of competition in the telecommunications 
industry raised issues about the maintenance, level, and redistri- 
bution of past subsidies among different telecommunications user 
groups and services. 


o Continued access to and the rates for telco local distribution 
plant, and the development of alternative local distribution 
telecommunications media seemed to be critical areas for further 
regulatory policy modifications to foster a pro-competitive 
environment. 


In substance, the report showed that no further monitoring was warranted 


in the limited field of PMS because this field had become virtually indistin- 


ee ee 


Le Derived “from Op. Cit., p- III-3, Executive Summary, p. i1. 
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guishable from that of computerized telecommunication services. Instead, a 
system was needed for monitoring and assessing the telecommunications industry 
generally. ETIP agreed to assist FCC in developing the capabiity for ongoing 
measurement of technological and competitive trends in the "store and forward" 
computer communications market, but in its final months ETIP’s objective prag- 
matically shifted toward the more modest goal of leaving behind within FCC an 
approach for developing and using information-gathering and asssessment systems 
as supports and guides in future regulatory decision-making. 

That new technologies exploded upon the message-transmission industry could 
hardly be contested, but it was just as true that such innovation could not be 
attributed to deregulation of PMS. Still, the PMS industry did witness the 
beginning of competition in what had formerly been a monopoly. At least two 
firms (Graphnet and National IMS) entered the field, offering in one instance 
lower rates than Western Union Telegrams and in the other speedier service than 
Western Union Mailgrams. By 1982 Graphnet was reporting operating revenues of 


$15 million. 


4. Marketplace and Government Results 

The link between ETIP regulatory activities and technological change in 
the private sector was admittedly weak. Although the experiments conducted 
were hypothesized to result in an improved climate for technological change in 
the private sector, with the possible exception of the FRA/ICC experiment with 
deregulation of fresh produce shipments and the subsequent use of advanced tech- 


nology by the railroad industry, the cause-and-effect linkages were lengthy and 


1. These concluding remarks are derived in part from Op. Cit., p. III-6, 
Kelly, pp- 78-79. 
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the time intervals over which industry could reasonably be expected to respond 
were greater than ETIP’s lifespan. 

The problem was not so much that the experiments had not stimulated new 
technologiese It was rather that, given its abbreviated history, little reason 
existed for expecting that ETIP could have determined what exactly were the 
effects of any of the modifications of regulatory processes on the rate and 
direction of technological innovation. 

The complexity of the experimental environment and the relatively small 
magnitude of the experimental modifications relative to some of the paldetead. 
and economic variables intervening in the regulatory process are two major pro- 
blems that regulatory change must overcome before conclusive policy ecdaumnads 
tions can be formulated regarding the acceleration and direction of technologi- 
cal innovation that might result from regulatory experimentation. 

What did result from ETIP experimentation with regulatory practices and 
policies in a number of instances was reduced costs to consumers. This was 
true both in the deregulation of the ist records message services industry by 
FCC and in an additional project (No. 76, not considered in this report) which 
helped State public utility commissions significantly reduce the time required 
for rate-setting by using data-analysis techniques. This time-lag reduction 
subsequently allowed the regulated utility companies to introduce cost-cutting 
technologies. 

More important than these marketplace results, however, was the raising of 
innovation-consciousness and the awakening of willingness to experiment with 
policy changes within the partner agencies with which ETIP worked. This con- 


sciousness-raising and the institutionalization of procedures to analyze the 
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likely impacts of current and anticipated regulations upon the course of inno- 
vation in the regulated industries occurred in virtually every agency with 
which ETIP’s Regulatory Program Area worked, most notably at FDA, FCC, ICC, EPA, 
and the State utility commissions. These were direct and visible changes 
wrought by ETIP that are likely to have beneficial effects on both government 


and industry long after ETIP itself is terminated. 


Be. PROCUREMENT 


The Federal government purchases an estimated $85 billion in goods and 
services annually. ETIP hypothesized that this immense purchasing power could 
be harnessed as an effective policy tool by providing new or increased demand 
for desirable technological change in those consumer and service industry goods 
of which the government was a major buyer. The government, however, had proven 
to be an uneducated bargain-hunter concerned almost exclusively with the immedi- 
ate price tag of each item it purchased: 

-»-(T)he government usually purchased the cheapest products that met 

minimal design standards. Not surprisingly, these products were also the 

simplest and most basic, incorporating technology that lagged significant- 
ly behind that available to private consumers. ETIP argued that if the 
government specified performance needs rather than design standards and 
purchased products with the lowest life cycle cost (initial price plus 
operating expenses), it would not only improve the quality of the goods 
purchased, but also, in the long run, save money. At the same time, this 
policy would encourage industrial innovation. 

ETIP identified three ways in which, if properly designed and implemented, 


government procurement activities could foster technological innovations and 


their diffusion into the marketplace: 


l. Op. Cit., p- I-2, Britan, p- 56-58. 


oO 
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The most direct impact government procurement can have is by apply- 
ing its buying power to create a market for products and systems 
beyond the state-of-the-art. There are two major categories of 
government purchases in this area: purchases of innovations for 
government ’s own use, as by the National Aeronautics and Space 
Administration (NASA) and DOD, and purchases of innovations for 
use by society at large, as with technologies for meeting regula- 
tory requirements imposed by such agencies as DOE, DOT, and EPA. 


Government procurement can create a demand "pull" for new techno- 
logies in the process of filling its routine requirements if it 
expresses its needs to industry in functional or performance terms. 
Such an approach would permit industry to exercise initiative and 
creativity in responding to these needs with innovative products and 
approaches. The size of the government market and its continuing 
nature would then serve as stimuli for industry to initiate develop- 
mental work. 


The government market also represents a sizable testing ground for 
innovative products. The ability to place new products in a field 
environment and monitor product performance can provide valuable 
information concerning product modifications, especially to regula- 
tory agencies (e.g.-, DOE, DOT, and EPA) which had congressional 
mandates that required product innovation to achieve national 
objectives. 


To attempt to harness these three procurement influences (market creation,+ 


demand pull, 


and prototype testing), ETIP identified innovative procurement pro- 


cedures and contractual provisions to be tested and the potential administrative 


"targets of opportunity" for experiments, including FSS, the Veterans Administra- 


tion (VA), and State local and governments. FETIP concentrated its initial pro- 


curement~area activities at FSS in experiments with the following innovative 


approaches to government purchasing: 


O 


Life Cycle Costing (LCC): Determining total costs incurred over 
the normal lifetime of product use and substitution these lifetime 


costs in place of the initial low bid as the criterion for procure- 
ment award. 


Value Incentive Clause (VIC): Stipulation in procurement contracts 
that the manufacturer may share in cost savings to the government 
that result from innovative, efficient designs by the manufacturer. 
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o Performance Specifications: Stipulation of improved product per- 
formance, rather than explicit design features, in describing 
government needs to industry- The manufacturer is encouraged to 
offer novel, low-cost designs so long as specified product perfor- 
mance standards are met. 

o Sliding Scale Rating: Used in conjunction with performance speci- 
fications, a technique that rewards manufacturers offering superior 
product performance by reducing their cost bid according to an 
established scale for purposes of determining the effective low bid. 

o Two-Step Procurement: A technique whereby manufacturers are asked 
to submit technical proposals (to establish the feasibility of 
innovation) prior to cost proposals. This technique was standard 
in the Federal Procurement Manual but was little used by many 
agencies. 

o Prototype Purchasing: A non-competitive means of allowing manufac- 
turers to sell a limited quantity of an innovative product on the 
assumption that successful testing and evaluation of the product 
would lead to a larger government market. 

ETIP’s Procurement Area actively worked to test these incentives in several 
agencies for three years. Thereafter, the focus shifted to the process of admin- 
istrative change within line agencies to establish a pattern of policy-~induced 
technology change. 

To conduct its procurement experiments ETIP developed formal cooperative 
agreements with FSS, VA, the National Institute of Government Purchasing (NIGP), 
and the National Association of State Purchasing Officers (NASPO). After a 
trial period of two years, the NIGP and NASPO arrangements were abandoned and, 
for a variety of reasons to be discussed later, the VA agreement proved ineffec- 
tive in either conducting procurement experiments or in institutionalizing pro- 
cedural change. The FSS agreement, however, succeeded in establishing LCC and 
VIC as routine procurement procedures. FSS Commissioner Mike Timbers, who began 


his tenure in June 1973, credited ETIP with a crucial role in getting the new 


new procedures adopted and standardized. The procurement agency now uses LCC 
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in many procurements, and internal policy statement requies that VIC be used 
in all contracts worth $100,000 or more. 

The following sections discuss results of ETIP experimentation with LCC 
and VIC, and of an ETIP procurement project that tested the degree to which 
government procurement could support government regulations while simultaneously 
improving the climate for technological innovation. Also included is discus- 
sion of a procurement experiment with VA and subsequent retargeting of efforts 
to mission-oriented agencies with built-in incentives to foster technological 


innovation. 


hs Life Cycle Costing (Project Nos. 36 and 63) 


The Life Cycle Costing (LCC) approach to procurement is intended to 
obtain the best performance for the lowest total cost of ownership. Although 
major benefit to be derived from this approach was to be a reduction in overall 
cost to the government, it was hypothesized by ETIP that LCC also would stimu- 
late among government suppliers a competition to lower the total-life costs of 
of their products by use of innovative technologies to lengthen the service life 
of products, to reduce their levels of energy consumption, to reduce maintenance 
costse etce 

It should be noted again that ETIP’s overriding purpose was not directly 
to bring about new technology but to gather information about those policies 
and mechanisms which might be expected to foster technological innovation and 
its diffusion. Nevertheless, individual projects did result in cost-savings to 
the government and in procurement of substantially improved, technologically 


advanced products. For example: 
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o Water Heaters with substantially reduced energy consumption were 
purchased by FSS using both LCC and performance specification 
approaches. They were subsequently sold commercially. A 
reliable estimate placed savings to the government at about $2.3 
million a year. 

o Air Conditioners procured using LCC had better energy efficiency 
than those procured previously using the lowest initial bid as 
criterion. 

These procurements were among a long list of appliances purchased by the 
FSS using LCC based on energy costs of operation. 

Another LCC project (No. 36) initiated by ETIP established for the Public 
Buildings Service (PBS) a comprehensive LCC approach to the analysis of options 
for meeting Federal space requirements. An LCC model was developed, computer- 
ized, and employed within PBS, but the relationship between the model and inno- 
vation was not identified, nor did the model establish a basis for policy recom-— 
mendations to PBS or other agencies. 

ETIP also took four practical actions to help agencies interested in the 
use of LCC: 

o It produced a "how to do it" manual; 


o It conducted training sessions; 


o It provided direct technical assistance for specific applications; 
and 


o It provided documentation of success stories that potential users 
could read. 


aes Value Incentive Clause (Project No. 64) 


The Value Incentive Clause (VIC) approach to procurement is intended to 
encourage contractors to share in cost savings accruing to government buyers 
from acceptable value change proposals (VCP’s) made by the contractor. VCP’s may 


affect costs associated with specification, design, material, process, tolerance, 
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packaging, or testing considerations, or product operation, maintenance, dispos- 
al, or replacement, as long as they directly benefit the government in the reduc— 
tion of overall cost without impairment of quality or performance. 

VIC had been used in other government agencies, notably DOD since 1962. 
ETIP and FSS decided VIC had considerable potential as a "best value" procure- 
ment technique and incorporated it into FSS purchasing in March 1974. 

The VIC was formally instituted as a contractual clause by GSA and attached 
to all contracts of $100,000 or more beginning in June 1975. To acquaint FSS 
and industrial personnel with VIC, FSS held orientation seminars, prepared and 
disseminated informational brochures, and developed two video-tapes explaining 
VIC. Personal contacts were also made with a sample of 50 contractors by the 
FSS commissioner in an attempt to encourage response. 

For this experiment the major intended impacts on industry from the FSS‘s 
perspective were to: 


o Increase the value-for-the-dollar of government purchases during 
a contract as well as with each new contract, and to 


o Encourage industry to recommend improvements in contract specifi- 
cations, thereby giving industry a specific way of participating 
in a process to help the government continuously upgrade its pro- 
curement specifications and processes. 

The major intended impacts on industry from ETIP’s perspective were to: 

o Provide an opportunity and incentive for companies to offer techno- 
logical improvements in products they already had contracts to 
supply, and to 

o Provide an incentive for the more innovative companies to bid (i-ee., 
to obtain a guaranteed market for new R&D, thereby reducing the 
risk of R&D investment). 

In FY 1976, the first full year of the project, an estimated 1,500 FSS 
contracts contained the VIC, but as of September 30, 1976, only 12 VCP’s had 


been received by FSS. ETIP contracted with SRI International to determine why 
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the response had been so low and to obtain specific information that would help 


improve the VIC program. SRI reported: ! 


oO 


Inclusion of the VIC in the contract and other promotional activi- 
ty still often did not result in...(a) firm’s awareness of the 
VIC option in an active contract held by the firm. 


Because of the complexity within FSS of using...(the) new con- 
tract vehicle, in the first year VIC was often not included in a 
contract and even more often included only by reference. 


‘In the relationship between FSS and industry, experience... (had) 


not yet instilled confidence in the firms...that a VCP submis- 
sion would result in a positive outcome. 


Many firms..-whose officials were aware of the option felt that 
the VIC was not applicable to their product and had not made any 
effort to explore the opportunty. 


Nevertheless, in several contracts where the VIC was successfully employed 


by contractors, SRI considered the results mutually beneficial: 


oO 


FSS implementation of three changes in a contract for computer-t ype 
air conditioning units that had been recommended by the contractor 
saved the government $394,145 in installation, power use, and elec- 
trical demand costs over a ten-year period. The contractor’s 20 
percent incentive award share of an average year’s savings added 
$7,882.90 to earnings on the eontract. 


A contractor-suggested alternatives for a more efficient compres- 
sor and other changes contributed to higher quality computer-room 
air conditioning systems. Total estimated savings to the govern- 
ment in maintenance, power use, and electrical demand for the com- 
pressors were $378,510.80 over ten years. The contractor’s 20 
percent incentive-award share of an average year’s savings to the 
government added $7,570.22 to contract earnings. 


The FSS implemented the major part of a VCP submitted by a contrac-— 
tor suggesting an innovative way of fastening the corners of 
household-goods shipping containers, thereby increasing their ser- 
vice life 400 percent. Ten-year estimated savings to the govern- 
ment exceeded $7-5 million- The contractor’s incentive award 
exceeded $150,000. 


l. "Commercial Impact Evaluation of ETIP Procurement Experiments, Phase 
I," Interim Report, SRI International, June 1978, p.- ll. 
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o A contractor’s suggested changes to GSA specifications for ward- 
robe hinges saved the government $1.56 per wardrobe purchased and 
won the contractor a 50 percent award of $780 in additional earn- 
ings on the contract. 

In the above cases, the government received considerable indirect savings 
as well as those listed, among them (a) updated specifications and concomitant 
continued savings for many years; (b) reductions in maintenance, operation, and 
disposal costs (as of scrapped shipping containers); (c) conservation of energy 
and other resources; and (d) reductions in future contracting efforts by 
requiring procurements less often in cases where the VCP’s resulted in longer 
product life. 

The cases also demonstrated "the capability of contractors to propose 
alternate methods for accomplishing a job in a more cost-effective manner by 
challenging government specifications which have become outdated due to advances 
in technology or improved methods and procedures. It also confirms the commit-— 
ment of the...-(FSS) to accept and implement good...(VCP’s) and to return a 
fair share of the savings to the contractor who invests in making the effort to 
improve the product.""! 

Overall, however, the VIC was not the stimulus to innovation that had been 
expected, probably because contractors considered the share of savings returned 
to them as insufficient incentive. As a result, only a small number of the 
potentially applicable VIC’s produced VCP responses, occasioning an expenditure 
of a great amount of effort for little gain. 

1. See “Summary and Conclusions" (p. 1) of "Value Incentive Program, 
Computer Room Air Conditioning Units," July 1976; "Value Incentive Program, 
Energy Conservation Compressors for Computer Room Air Conditioning System," 
October 1976; "Value Incentive Program, Household Goods Shipping Con- 
tainers," July 1976; and "Value Incentive Program, Reduce Welding/Brazing on 
Wardrobe Hinge," October 1976: Final Reports prepared for ETIP by Kempter-— 


Rossman International, Washington, D.C. (NBS-GCR ETIP 76-23, 26, 18 and 17 
respectively.) 
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3. Performance Specifications 

Another way, it was believed, to enlist government procurement muscle in 
the service of stimulating private-sector technological innovation was to apply 
performance specifications (as opposed to the more traditional detailed product- 
design specifications) in RFP’s that would dovetail with and otherwise buttress 
national policy on pending or forthcoming standards and other regulations. A 
case study that best illustrates the attempt to test this hypothesis, the pro- 
blems encountered, and the chances for success was the procurement of low-noise 
lawnmowers by Fss./ 

Power lawnmowers were chosen for the experiment because they were bought in 
large quantity, could be readily transferred to commercial markets, and were the 
subject of growing national concern regarding safety and noise. 

In designing the experiment ETIP and FSS worked closely with EPA and the 
Consumer Product Safety Commission (CPSC), each of which was responsible for and 
interested in new regulations for lawnmowers: EPA for noise, CPSC for safety. 
Innovations were sought to reduce noise to 85 dB(A) from the standard industry 
range of 86-92 dB(A), and to introduce a "dead man’s clutch," a safety device 
which would stop the mower’s cutting action within three seconds after an opera- 
tor released the handle. 

The project plan elaborated the focus of this objective in its statement of 
evaluation criteria: 

The procurement approach will be judged effective...if and when 


improvements traceable to that mechanism are regularly offered in 
the private sector of the marketplace. 


1. This case study is drawn in part from Op. Cit., p.- III-6, Kelly, 
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It is evident from this that ETIP was interested principally in technology 
change rather than agency administrative change, at least at the launching of 
this experiment. 

Neither EPA nor CPSC formally agreed to delay publication of new standards 
until results of the experimental procurement became available, but there was 
reason to believe some manufacturers perceived a relationship between the FSS 
lawnmower performance specifications and the performance standards being tar- 
geted by EPA and CPSC. 

In September 1975, FSS issued a letter of intent to purchase 10,000 lawn- 
mowers. Response was poor because industry was unable to solve the problem of 
a cheap, reliable clutch within the required time. This was in spite of CPSC’s 
1974 announcement of its intention to issue requirements for a "dead man" 
feature. 

FFS specifications were issued that eliminated the "dead man" feature (but 
retained the noise-abatement feature) because CPSC stated that its safety stan- 
dard was at least a year away. The on .erre eran set the maximum dB(A) at 86, 
established a sliding scale whereby manufacturers offering 82 dB(A) would 
receive a $10 advantage in determining the low bid, and limited proposals to 
4-cycle engine models, those most popular among industrial and other large-scale 
consumers. 

Eight technical proposals were received. All were acceptable technically, 
but one was eliminated because it had been received after the deadline. The 
contract was awarded to AMF’s Lawn and Garden Division of Des Moines, Iowa, in 
February 1977. When FSS tested a prototype of the AMF mower, noise emission 


levels of less than 80 dB(A) were recorded. 
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Significantly, the runner-up was a lawnmower also to be made by AMF which 
proposed noise emissions of 86 dB(A) and which cost $9.95 less per unit than 
the winner. The $10/unit bonus for improved noise-emission performance deter- 
mined that the low-noise machine would be considered the lower bid by a margin 
of five cents per unit, a ploy AMF officials later frankly called "smart 
pricing." 

In June 1977, GSA issued a news release implying that consumers would soon 
benefit from the availability of quieter lawnmowers, but AMF admitted it would 
not have manufactured such a mower had it not been for ETIP/FSS and questioned 
its marketability because it would cost more than comparable, noisier mowers 
and could not accept a grass bag attachment. AMF had no plans to market the 
machine. 

As part of its evaluation of the project a DOC team headed by Thomas V. 
Kelly! interviewed officials of Seiffert Manufacturing Company of Kiel, 
Wisconsin, whose bid had been disqualified for tardiness, because its proposal 
had included a unique, workable "dead man’s clutch" and a combination of noise 
emission (under 80 dB(A)) and cost ($82 per unit) which would have won the con- 
tract had it not been disqualified. (ETIP and FSS subsequently cooperated on 
purchase of 50 prototype Seiffert units to determine whether they met the 
technical claims made in the Seiffert proposal.) 

A principal implicit incentive for industry response to this procurement 
had been the threat of regulation. Industry representatives had been resisting 
lower noise standards on the grounds that marketing experience demonstrated con- 


sumers wanted cheap machines that cut grass and were unwilling to pay extra for 


1. Ibid. 
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noise reduction. EPA felt, as one manufacturer later corroborated, that a 
strong incentive in the ETIP/FSS experiment would be to provide a profit motive 
(10,000 lawnmowers x $10 bonus per unit) for industry to invent a substantially 
quieter lawnmower. 

Seiffert, however, described its incentives quite differently. A small 
firm which manufactures and assembles lawn and farm equipment for other com- 
panies, it had no marketing network of its own but aspired to develop one. Six 
years earlier, when lawnmower safety had become a public issue, Seiffert had 
begun work on a “dead man’s clutch" but shelved it because of difficulty in 
getting product liability insurance and because CPSC in 1973 condemned walk- 
behind rotary mowers as unsafe and threatened action. Seiffert was aE GADeT 
short by regulatory doubt and concern that its device might work on a different 
principle than that which ultimately would be acceptable to CPSC. 

When in 1976 Seiffert heard of ETIP and understood that ETIP had the 
blessing of CPSC, the manufacturer saw its chance to demonstrate to CPSC its 
invention in a powerfully persuasive way Sucve: simultaneously gaining momentum 
toward a marketing capability of its own. By coincidence, the design of the 
clutch which it had patented made the achievement of noise reduction a relatively 
simple and inexpensive modification. 

Seiffert stressed that the indispensible factor was protection of proprie- 
tary rights. Had there been any chance that by responding to this procurement 
Seiffert would have lost its exclusive patent rights, its new lawnmower would 
have stayed on the shelf. Combining the opportunity to anticipate regulatory 
decision-making, initiate a market, and compete confidently for a short-range 
profit with the essential condition of protecting proprietary rights proved an 


irresistible combination of incentives. 
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It should be noted that CPSC subsequently issued proposed safety stan- 
dards for walk-behind push rotary lawnmowers which incorporated a requirement 
for a "dead man" feature. Although the standard was published after the results 
of the ETIP/FSS experiment were available, CPSC officials disclaimed any real 
influence by ETIP other than adding marginal assurance to its belief that a 
cheap safety clutch was feasible. 

As to noise-abatement standards: 

As long as noise control...was a high priority at EPA, the poten- 
tial for a multi-product experiment was good and its potential value 
to EPA was high. When priorities switched to urban noise, Federal 
procurement experiments became somewhat less valued, and essential 
momentum in the idea was lost. With the shift in focus to NIGP and 


State level involvement, new actors/stakeholders became involved, 
and the decentralized experimental process soon followed. 


4. Veterans Administration (Project No. 80)? 

ETIP Project No. 80 was not a single experimental project as such but an 
agreement between ETIP and VA to undertake experimental procurement activities 
for the VA medical system. The interagency agreement (June 1975) had the 


following objective: 


To implement and institutionalize within the VA Supply Service (VASS) 
the ETIP philosophy of using procurement as an incentive to stimulate 
innovation and technological change in the nation’s health care 
delivery system. 


Cooperation with the VA seemed a natural extension of ETIP’s work with FSS. 
While FSS acts as a purchasing agent for a separate user population, the VA is 


both purchaser and user of medical devices. 


1. "Procurement Experimentation in Federal Agencies, Phase III Final 
Report," Research Triangle Institute, May 1981, (21U-1689-FR) p. 44. 

2. This section is drawn in part from Op. Cit. p. III-6, Kelly, pp- 
107-116. 
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Following are examples of activities by the Experimental Procurement Office 


(EPO) established by the experiment: 


oO 


Oo 


Portable oxygen generator: This was an attempt to use performance 
specifications to spur a major technical advance in an extremely 
complex piece of equipment. It was unsuccessful except that one 
firm’s proposal offered serendipitously a demand cannula (nasal 
breathing device) previously unavailable, which VA will purchase 
separately. 


Urine specimen container: Performance specifications were used to 
obtain a modification of a simple non-mechanical device. Although 
the procurement was successful, it was not clear that VA hospital 
demand would put it to wide use. 


Catheter cabinet: This was an unsuccessful attempt to convince 
industry to manufacture a specially designed cabinet which was 
devised in one of the VA hospitals. 


The problems affecting ETIP activity within VA stemmed from many sources -- 


some structural, some conceptual: 


Understaffing: EPO had at most only two staff members, one in Hines, 


Illinois, the other in Bay Pines, Florida, who quit the project in 1977. It 


is doubtful that one or even two persons could expect to have had significant 


impact on a system as large as that of VA. 


Management Attitudes: High-level VA officials were tolerant but largely 


unimpressed by the efforts of ETIP in their agency. As time passed and few 


results were noted, managers told a DOC evaluation team they saw no essen- 


tial difference between the ETIP-sponsored EPO and the VA Testing and Evalua- 


tion Staff in which EPO was placed. Free staff and money were seen as the main 


benefits of ETIP intervention. While no animosity toward ETIP was expressed by 


VA officials, they did express an impatience with ETIP for not better defining 


its mission in VA. 


1. Ibid. 
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Conceptual Ambiguity: One VA staff member expressed the view that ETIP 
started out backwards, concentrating on testing procurement methods rather than 
on identifying widely useful product innovations. As a result, time and effort 
had been lost experimenting with the standard but little used two-step procure- 
ment technique when there was no assurance that the product being solicited 
would be widely used even if it had become available. 

In the case of the failed catheter cabinet procurement attempt, the typical 
ETIP approach of establishing a need for new technology and then experimenting 
with incentives to get manufacturers to produce an acceptable product innovation 
was reversed. In this case the innovation existed within VA and manufacturers 
were asked to produce it for markets which might or might not have existed. 

Nature of the User Group: The VA hospital system is a loose confederation 
of semi-autonomous institutions staffed by medical and support professionals. 
The traditional independence of physicians as a professional group found its 
reflection in the resistance of hospitals to standardized equipment. Because of 
this, ETIP’s operating assumption that VA ee as a single buying/using organi- 
zation was in error. Many hospitals maintained their own workshops and con- 
structed equipment of a design satisfactory to their own needs. Transfer of new 
designs from one hospital to others was infrequent and almost a majority of 
medical staff was invariably dissatisfied with anything VASS offered. 

Industry Suspicion: Manufacturers were reluctant to devote resources 
to developing new products primarily for VA use because of the problem of pre- 
dicting system demand and because field tests were required for every new device 


before an order could be placed for more than the initial test quantity. Thus, 
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VA could not guarantee a manufacturer that it would ever order a sufficient 
quantity of a new item to repay the manufacturer’s R&D investment. 

Before EPO could overcome industry’s distrust it needed to devise an 
effective demand information system for medical supplies. The practical 
consequence of this was to force ETIP into the role of market research con-— 
sultant for a closed-system Federal agency. Such a service by ETIP, however 
beneficial to VA and to ETIP in establishing the internal structure assumed in 
the initial decision to work with VA, was too removed from ETIP’s missions as 
student and catalyst. 

The VA/ETIP experimentation ended informally in 1979. It had proven — 
frustrating and disappointing for both parties. With one minor exception, no 
experimental procurements resulted in innovative products, and it was openly 
questioned whether the learning process had been sufficiently illuminating to 


justify the interest and energy expended. 


Yr) Mission—Agency Strategy 


Based on its first several years” experience, ETIP recognized that procure- 
ment agencies such as FSS and VA do not have the marketplace leverage hypothe- 
sized nor the ongoing incentive to focus on technological innovation unless it 
results directly in savings. Budget-consciousness forced such agencies to seek 
the lowest bid possible first; technological innovation could at best be an 
ancillary benefit. 

Because of these considerations ETIP moved the locus of procurement experi- 
ments into the larger, more mission-oriented agencies where there existed 
greater marketplace leverage and greater incentive to stimulate technological 


advance. Contracts were initiated with DOD, DOE, and EPA. 
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Examples of these subsequent efforts by ETIP and its procurement 


experimentation contractors, SRI International and Research Triangle Institute 


(RTI), include: 


o Major new technology purchases by DOD, such as of a new generation 
of Very High Speed Integrated Circuits, which bought into play 
"outside" firms and demonstrated the importance of major-user needs 
in fostering private-sector inventiveness; 


o The use of cooperative agreements in DOE-spurred development of 
energy-conservation technologies (e.g., solar energy cells); and 


o Prototype purchasing and testing by DOE (solar cells), the Postal 
Service (electric cars), and EPA (furniture coatings that use a low- 
emission water solvent process). (This latter demonstrated the 
use of government’s procurement power to speed the achievement of 
regulatory goals by providing regulators with feedback on techni- 


cal feasibility and reducing the level of risk to developers of 
the new technologies.) 


6. Marketplace and Government Results 

The marketplace spin-offs of ETIP’s procurement experiments were admit- 
tedly modest. From the examples considered in this chapter, these included 
energy-saving appliances; cost-saving innovations in air conditioners, ship- 
ping container corner fasteners, and wardrobe hinges; and the Seiffert 
lawnmower. 

More importantly, ETIP’s procurement area initiatives significantly 
altered government agency perceptions and formalized procurement strategies de- 
signed to encourage development of advanced technologies. 

o OMB’s Circular A109 put the government on record as favoring use 
of performance specifications in describing its system require- 


ments to suppliers; 


o OMB’s Office of Federal Procurement Policy (OFPP) adopted the per- 
formance specifications strategy for product procurements; 


o FSS institutionalized a range of innovation-encouraging procurement 
strategies, including LCC, VIC, and performance specifications; and 
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o A number of agencies, notably EPA, DOE, and DOT, actively began 
pursuing procurement strategies to foster technological advances 
calculated to meet regulatory objectives. 

(Although ETIP cooperated actively in DOD procurement initiatives, it could 
not take credit for ongoing, institutionalized strategies which had been used by 


DOD as early as 1962.) 


C. ECONOMIC ASSISTANCE 


During ETIP’s first three years of operation Small Business and Research 
and Development were combined within a single program area that emphasized the 
R&D stage of the innovation process. It was hypothesized that small businesses, 
particularly those that were technology-based, faced special difficulties in 
attempting to develop new technologies and survive economically. ETIP, there-— 
fore, selected to examine government’s role in either adding to or alleviating 
those difficulties. 

By 1976 it became clear that the eedeene of marketing the products of R&D 
should also be exmained to develop an understanding of the leverage various 
government policies could be expected to have at the various stages of the 
innovation process. R&D was separated out into a distinct program area and 
small business-oriented initiatives were incorporated within a new Economic 
Assistance Program Area whose mission was defined more broadly. 

Whereas the Regulatory Area targeted specific attempts to lessen the nega- 
tive impacts of government regulations on private-sector innovation, and Pro- 
curement Area experiments attempted to exercise the muscle of government pur- 


chasing to stimulate such innovation, Economic Assistance Area interests cut 
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across a range of government agencies and activities including direct aid by 
infusion of money and other assistance into the private sector (e.g., loans and 
subsidies), regulatory and other government actions believed to affect innova- 
tive firms and private infusion of money into them (e.g., securities regula- 
tions), and program options which could affect private sector initiatives in 
support of national policy in response to (or in anticipation of) international 
marketplace upheavals (e.g., countermeasure options for commodity supply 
shortfalls). 

In this area ETIP functioned more as a student/researcher of what 
government can do than as change agent, although several of its studies did 
result in modified line-agency policies and procedures. 

The following ETIP activities are considered in this section: 

o Analysis of SBA mechanisms that could be used to facilitate private 
sector development and/or adoption of new technologies to meet 
regulatory requirements; 

o Analysis of venture capital markets to determine whether a con- 
sistent pattern of under-funding of small technology-based firms 
existed, and development of a system to monitor for these and 
other effects arising from SEC regulations; and 

o Funding and monitoring of an innovative program (the Connecticut 
Product Development Commission) to evaluate its effectiveness as 
a mechanism for injecting capital into private-sector technological 
development and to generate guidelines for future similar efforts. 

This section examines these ETIP efforts in terms of what changes they 
effected in the marketplace and/or in government agency policies and procedures. 
The generalizable lessons derived from these studies and from others conducted 
by this program area (a case study of the semiconductor industry to examine the 


dynamics and effectiveness of a range of government policies; analysis of a 


variety of policy options to meet commodity shortfalls resulting from interna- 
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tional marketplace upheavals; and assessement of the efficacy of direct aid to 
private sector R&D through the granting of capital subsidies) will be treated 


in Chapter IV. 


it Regulatory Impacts on Small Businesses (Project No. 28) 


Between 1974 and 1976 ETIP sponsored a series of studies in cooperation 
with SBA to examine how government could best support small, technology-based 
companies generally considered to be important sources of innovative products. 
This section considers the most significant of these studies. 

Project No. 28 was a two-year effort by Charleswater Associates, Inc., 
(CAI) to identify and critically examine actions SBA could take to facilitate 
development and/or adoption of technologies forced on small businesses by govern- 
ment regulations and otherwise to alleviate problems small businesses face in 
complying with such regulations. CAI’s final report! cited several difficul- 
ties small businesses face in complying with regulations. These included: 

o Fragmentation of regulations which affect small businesses; 


o Problems raising capital for "non-productive" investments in com- 
pliance technology; 


o A tendency to seek "end of pipe" rather than "process" solutions; 


o Difficulties in realizing economies of scale in complying with 
environmental standards; and 


o Lack of an effective "voice" in Washington. 
CAI made the following recommendations: 

o "The single most cost-effective step" SBA could take is to "inter- 
vene in the Federal legislative and regulatory process as an active 
but objective advocate of small business needs and interests." 

1. "The Impact on Small Business Concerns of Government Regulations That 


Force Technological Change," Charleswater Associates, Inc., September 1975 
(GPA Stock No. 045-000-00144-0). 
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o SBA should "expand its financial assistance activities (a) by 
aggressive use of its authority to provide comprehensive Regula- 
tory Economic Injury Assistance loans, (b) by providing loan appli- 
cants with technical support for the preparation of applications, 
(c) by expanding its lease guarantee program to include guarantees 
for leasing the equipment and facilities necessary for regulatory 
compliance, and (d) by providing guarantees for bonds issued by 
small businesses." 


o SBA should "revive and expand" its technical assistance program by 
(a) maintaining a file of "as built" descriptions of compliance 
systems, (b) conducting management seminars on technical compliance 
problems, and (c) providing direct applications assistance. 


o SBA should foster cooperative action among small businesses and 
encourage (a) collaborative approaches to research and development 
of solutions to compliance problems, (b) jointly-operated produc- 
tion and pollution control facilities, (c) joint purchasing of 
materials and services to obtain the price advantages of mass 
buying power, (d) joint representation before regulatory agencies, 
elected officials, and the public,,and (e) institution and defense 
of legal actions on a joint basis. 

As a result of these findings SBA increased its intervention in regulatory 
rule-making processes by upgrading and expanding its office of Chief Counsel for 
Advocacy and began working more closely with several regulatory agencies. Among 
the results of SBA and ETIP initiatives: 

o The Occupational Safety and Health Administration (OSHA) offered 
assistance and advice to small businesses for meeting OSHA 
regulations, and 

o EPA promoted among small businesses SBA’s Economic Injury Loans 
Program, which provides relief to small firms suffering substantial 
economic injury as a result of environmental regulations. 

Additionally, SBA institutionalized several other CAI recommendations. It 
began providing technical assistance to small businesses to enable them to 


understand better the technical requirements of new regulations, and its Techno- 


logy Utilization Program established an inventory of available technology to 


ee Ibid., ppe 7=-9. 
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meet those requirements and began transferring this information to small 


businesses. 


26 Venture Capital and SEC (Project Nose 9 and 148) 


Technology-based firms comprised an increasing proportion of company 
acquisitions being reported by U.S. corporations in the 1970s. Data from the 
Federal Trade Commission (FTC) suggested that these firms were unusually attrac- 
tive targets for larger acquisition-minded companies. While the total number of 
mergers was declining, the number of technology-based acquisitions increased 
sixfold from 1972 to 1977, and the ratio of these acquisitions to all completed 
acquisitions increased 1840 percent. This raised serious concern about the 
reasons for such mergers and their possible implications for regulatory policy 
with regard to competition and efficient resource allocation in both product and 
capital markets. 

The relative inability of private investors to gain liquidity in public 
capital markets led an SBA task force to suggest that 

large companies (are) able to entirely buy out successful small com- 
panies at a discounted price because the business...(has) little 
alternative in meeting (its) financing and liquidity needs. This is 
»eethe major force increasing concentration.... 

Concerned that these perceived inequities and imperfections in the venture 
capital markets appeared biased against technology-based small firms, in 1974 


ETIP funded a study by Charles River Associates (CRA) to analyze capital mar- 


kets and determine if there appeared to be any consistent patterns of underfund- 


1. "U.S. Small Business Administration, Report of the SBA Task Force on 
Venture and Equity Capital for Small Business," January 1977, p- 19, as quoted 
in an SEC Draft Paper, "The Acquisition of Technology-Based Firms in the OTC 
Market," A.E. Osborne, Jr., March 16, 1979, pe 4. 
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ing of small technology-based firms. Small firms in seven technology-based 
industries were examined: drugs; office and computing machines; radio, televi- 
sion, and communications equipment; electronic components and accessories; air- 
craft, guided missiles, and parts; scientific and mechanical measuring instru- 
ments; and optical, medical, and ophthalmic goods. CRA reported: ! 
We found no evidence of substantial market imperfections that restrict 
the flow of funds to small technology-based firms. We found no indi- 
cation that small technology-based firms paid higher rates of interest 
or returned their unaffiliated stockholders more than other small 
firms. We found no indication that suppliers of funds to small tech- 
nology-based firms earned higher profits than could be earned by 
investing in listed securities. We found no elements of the struc- 
ture of behavior in the venture capital industry which would enable 
firms in the business to charge higher rates or earn greater profits 
than necessary to compensate for the risks assumed. 

CRA added a caveat, however: failure to find imperfections did "not 
necessarily imply that the supply of funds to new technology-based firms (was) 
in some sense ideal" (p. S-2). Securities and tax regulations, CRA noted, 

did influence the rate of return required by suppliers of funds to those firms, 
and such regulations did "affect the profitability of small firms in general" 
(p- S-2). 

CRA found, however, that regulatory changes that had been proposed could 
not be expected to increase the flow of funds to small technology-based com- 
panies by more than 10 percent, and that loan or investment guarantees to sup- 


pliers of funds or directly to small firms (also being proposed) could similarly 


increase the flow of funds by no more than 10 percent. CRA concluded: 


1. “An Analysis of Venture Capital Market Imperfections, Executive 
Summary," Charles River Associates, February 1976 (NBS-GCR-ETIP 76-08) 
pp- S-l, 2. 
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-e-(A)ttempts to increase the flow of funds by direct government 

investment in small technology-based firms may tend to displace 

private funds rather than increase the total flow of funds (p. S-3). 

The CRA report established ETIP credibility that opened the door to a joint 
ETIP-SEC experiment (Project No. 148) begun in 1977. 

Both the Congress and SEC were already predisposed to do something about 
the burden of SEC regulations on small businesses when the project was initiated. 
In effect, ETIP did not so much “intervene” as present itself "at the right place 
at the right time." The project was two-phased. The first phase: 

o Developed a quantitative model for estimating and evaluating the 
economic impacts of SEC regulations specifically directed at or 
having pronounced effects on the venture capital market; and 

o Performed initial tests of the model by applying it retrospec-— 
tively to existing regulations and on a real-time basis to both 
new and existing regulations so that its usefulness as a policy 


decision-making tool can be assessed. 


The second phase: 

o Extended evaluation of the monitoring system’s efficacy by 
applying the model to several new or changed regulations over a 
sufficiently long period of time to allow measurement of all 
effects of securities regulation; and 


o Institutionalized the monitoring system within the SEC’s policy 
decision-making structure. 


A number of studies were made of SEC policies (e.g., Rule 144, Regulation 
A) and general areas of concern (e.g., mergers and short sales) to identify 
venture capital problems where monitoring might be done. An economic input 
analysis model and the data bases needed to run the analyses were developed. 
In one case the need for data resulted in changes on an SEC form for small 


firms, and testing of the system contributed to deliberations over changes to 


Rule 144 that were favorable to small firms. 
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Rule 144 set minimum holding period requirements, restricted resale volume, 
and required resale notices when securities acquired in a transaction which did 
not involve a public offering were intended to be resold. It was an element of 
the disclosure system SEC administers as part of its effort to protect investors. 
Using the developed market model analytical system, the joint ETIP-SEC project 
monitored the effects of the rule after amendments in 1978 and 1979 relaxed its 
resale restrictions somewhat. Results of the monitoring provided an empirical 
basis for further amendments in 1981 when it was shown that the burden of filing 
Rule 144 notices might be lifted without adverse effects on the capital market 
environment for small, venture capital-backed, high-technology firms. 
In Phase II several lines of work were developed: 
o Institutionalization of the system, 
o Closer ties to other SEC units, 
o Increased attention to impacts other than issue price, and 


o Broadening of the scope of the system to include other important 
stakeholders in the venture capital environment. 


The Phase II monitoring system focused on policy—relevant issues affect- 
ing firms of all sizes, but its primary intention was to compare the effects 
of SEC regulatory activities on the smaller, innovative companies with effects 
on all companies. Thus ETIP’s objective, the gathering of policy-relevant 
information about the securities markets confronting small issuers, was preserved. 
The computerized data base and library and the skills to use it were being 
fully institutionalized in 1982 as an integral part of SEC’s rule-making process 
and had already been employed successfully in monitoring for the effects of 


several information-gathering forms and regulations. Additionally the model had 
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been expanded to include other, non-SEC data bases, including the Federal Reserve 
System’s Flow of Funds data. 
In its final report on the project, SEC praised the program’s accomplish- 
ments and institutionalization: 
The versatility of the monitoring capability is-..an important 
characteristic, since successful institutionalization of empirical 
analysis at the Commission requires the timely presentation of useful, 
policy-oriented information as often and on as wide a variety of 
regulatory issues as is possible. The individuals who are the 
"clients" of the system, primarily the Commission itself and the 
senior staff of the Commission’s various divisions, change periodi- 
cally, and thus the status of the monitoring capability should not be 
dependent on particular individuals. 
To be effective, the monitoring system not only had to collect and apply 
correctly the relevant economic data and tools, but had also to develop channels 
of communication within SEC and see to it that the results of economic impact 


analyses reached decision-makers in an understandable and usable form so as to 


have an effect on decision-making. This was successfully accomplished. 


3. Connecticut Product Development Corporation (Project Nos. 32, 78, and 136 
Another set of venture capital projects conducted by ETIP consisted of 
fact-gathering by Charleswater Associates, Inc. (CAI) that combined elements 
of economic aid to small business and research and development. 
The Connecticut Product Development Corporation (CPDC) was the first 
attempt on the part of a quasi-public body in this country to inject what are 
essentially venture capital funds into product development. CPDC was to finance 


development of new products with state bond funds and to receive payback 


1. "A Monitoring System for Effective Regulation of Venture Capital 
Markets, Final Report," U.S. Securities and Exchange Commission, Directorate 
of Economic and Policy Analysis, October 23, 1981, p. 15. 
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through royalties on sale of successfully completed and marketed products. Thus, 


CPDC was "to fill a perceived imperfection in the capital market by providing 


financial assistance when none was available from the private sector."* 


Stymied in its earliest years by State administrative delays and court 


challenges, CPDC sought funding from ETIP, which agreed to contribute $300,000 


for investment in projects in return for access to CPDC records and clients for 


five years. 


Following is a sampler of CPDC-funded projects monitored for ETIP: 


oO 


Development of a machine to count and sort paper currency 
according to denominations; 


Development of an inspection system using a laser beam to 
scan bearings during the quality control phase of produc- 
tion; 


Development of concrete railroad ties for sale to Amtrak for 
its northeast corridor renovation; 


Development and marketing of a line of cookware designed for 
use in microwave ovens; 


A failed attempt to develop a."surgical zipper" -- a suture device 
designed to be applied to a patient’s body prior to surgery in 
order to reduce resulting scar tissue; and 


An attempt to market a diesel engine fuel trimmer to reduce marine 
engine fuel consumption by a lengthy sea voyage aboard a prototype 
ship that ended when the ship sank in a storm at sea and both crew 
members, including the company president and prime mover of the 
project, drowned. 


CPDC was monitored from 1975 to 1979 by CAI to determine the operational 


viability of the organization in its natural environment. Emphasis was placed 


on defining issues, problems, and possible application of the concept to other 


areas. CAI’s final report, submitted in March 1980, included a number of 


1. "Monitoring of Connecticut Product Development Corporation, Final 
Report," Charleswater Associates, Inc-, March 1980, p. 10. 
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guidelines for CPDC-like operations. Those interested should refer to the final 


report of the project (see footnote on preceding page). 


4. Marketplace and Government Results 


None of the Economic Assistance Area projects could be said to have direct-— 
ly generated product innovations or their diffusion in the marketplace, with the 
possible exception of those products invented and marketed with the aid of CPDC 
and, indirectly because of its partial funding of CPDC, ETIP. 

Still, the Economic Assistance Area did succeed in institutionalizing ETIP- 
like processes and perceptions within both SBA and SEC. Policies and procedures 
were modified and it was generally believed that the modifications improved the 


environment within which private-sector innovation takes place. 


De RESEARCH AND DEVELOPMENT 


The most direct way in which the Federal government influences the course 
of technological growth has traditionally been through programs in support of 
research and development (R&D) and their concomitant demonstration projects. 
Thus, ETIP’s efforts to examine and intervene in these Federal activi- 
ties represented its single attempt to modify and experiment with government 
processes that could directly impact the process of technological innovation in 
the private sector. 

The February 1974 Program Plan listed three questions to be addressed: 

o What are the criteria for Federal funding of civilian R&D? 


o What R&D partnership arrangements involving Federal funding 
speed the flow of R&D results into wide commercial use? 


o How can Federal R&D funds catalyze industrial R&D? 
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To answer these questions ETIP entered into two types of activities -- 
studies of the R&D process to derive answers to the first question, and specific 
R&D projects to test hypothetical answers to the latter two. The studies and 
their findings will be considered in Chapter IV. This section focuses on two 
specific R&D projects: 


o A demonstration project to spur commercialization of a known tech- 
nology (Integrated Utility Systems); and 


o Funding of R&D that applied consortium techniques to an attempt to 


develop a commercially practical technological innovation (fabric 
flame-retardants). . 


1. Integrated Utility Systems (Project No. Gay 


The basic concept of Integrated Utility Systems (IUS) is that integration 
within a single unit of five utility subsystems (electric power generation, 
heating and cooling, solid waste disposal, sewage treatment, and water supply) 
permits recycling of energy and materials that result in cost savings, fuel and 
water conservation, and increased environmental protection. 

In 1973 the Office of Facilities Engineering and Property Management 
(OFEPM) within HEW began considering large-scale IUS applications for medical 
and educational complexes. Lacking implementation funds, it sought them from 
ETIP. 

The project’s purpose was to provide an experimental environment in which 
the goals of private enterprise might be aligned with the goals of a specific 
Federal program office to achieve: 


o A demonstration of the efficiency IUS in one major area of 
application; 


o Specification of appropriate changes in perception, custom, and 
law which presently act as disincentives to commercialization 


of IUS technology; and 


l. This section is drawn from Op. Cit., p. IITI-6, Kelly, pp. 153-169. 
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o The capacity and incentives within the participating private firms 
to pursue LUS installations independent of further Federal initia- 
tives. 


To realize these objectives, the plan proposed the hiring of an applica- 
tions contractor "having an interest in commercializing IUS technology." 


The contractor’s willingness to cost-share would be one important 
criteria (sic) in the selection process. Cost-sharing implied that 
the contractor’s firm cannot profit from the architectural and 
engineering (A&E) effort alone, and that his incentive to partici- 
pate will be derived from his perception of the future market for 
IUS hardware. 


Contractor cost-sharing was vital to ETIP’s initial interest in the pro- 
ject because traditional government-funded demonstration projects too often had 
resulted in commercially impractical hardware that was not subsequently trans- 
ferred to the private sector. The cost-sharing provision was dropped, however, 
when the initial RFP drew disappointing response because, it was pointed out, 
cost-sharing in effect required that the contractor work without pay and because 
it violated engineering’s professional code of ethics against fee~splitting. 

Shifting the project’s focus occasioned an internal ETIP correspondence 
(May 6, 1975) in which the ETIP project monitor, Richard Penn, explained what 
ETIP stood to gain from the activity as modified: 


---(In general when the Federal government wanted to induce 
innovation in the non-governmental sector it has traditionally 
provided all or at least a large faction of the funding needed 
for the first use of the (new in context) technology. Here we 
propose only to fund the development of what we believe will be 
a sound (and convincing) argument that will lead to a decision 
to proceed on an innovative course by a university or medical 
complex. The government has pointedly told potential hosts that 
they should not look to the Federal government for any funding 
for the actual construction. If the decision to proceed is made, 
then it will be shown that the Federal government can cause an 
innovation at low cost to the government where the ecnomics pro- 
vide a clear case to the user to innovate. What we are doing is 
reducing the front-end cost to the user to reduce his before- 


decision outlay where there is perceived risk to the user because 
it has never been done before. 
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Instead of "information intervention" being used to supply information to 
industry so that the IUS technology would be "pushed" through private-sector 
marketing, in the experiment’s reformulation information from the feasibility 
studies was intended to stimulate demand for the technology on the part of 
potential users. If the stimulated demand was sufficient for one or more of 
the potential sites to build an IUS, then the government would have effectively 
"demonstrated" the technology for only the cost of the feasiblity study. ETIP 
would have acted as a "facilitator" rather than the traditinal government role 
of direct funder of R&D. 

The University of Florida and Central Michigan University were selected 
as host sites for the feasibility studies, and Reynolds, Smith and Hills (RSH) 
was chosen as applications contractor. Both universities made "good faith" com- 
mitments to proceed with design, construction, and implementation of the IUS 
installations if the feasibility studies demonstrated the systems” economic and 
technical viability. Following successful feasibility studies, both universi- 
ties also decided to seek funding for Beene bat 

Additionally, the project was extended to include an IUS feasibility study 
(also successful) for the City of Burlington, Vermont, which includes both a 
university and a major medical center. This was a deliberate attempt by ETIP 
to test the usefulness of such feasibility studies in more complex organization- 
al environments. 

ETIP’s IUS project could not yet be said to have directly stimulated 
diffusion of a new technology. At ETIP’s termination, construction had not yet 
begun at any of the three proposed sites, although the city of Burlington had 
provided $9 million in municipal bonds for the project and appeared prepared 


to move toward actual construction of the IUS facility. 
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An initial aim of the project was to develop "the capacity and incentives 
within the participating private firms to pursue IUS installations independent 
of further Federal initiatives" (June 1974 Project Plan, cover letter to 
Assistant Secretary Ancker-Johnson). Although it did not seem to occur as a 
result of the incentives included in the original plan, several individuals 
and contractors have since been promoting and/or marketing the IUS concept. 

In summary, although the IUS project had not yet led to actual imple- 
mentation, it contributed significantly to information about and promotion ~ 
of IUS technology. In view of the long lead times often required by projects 
of the magnitude of an IUS installation, this could be considered a significant 


contribution. 


2 Flame-Retardants for Polyester/Cotton Blends (Project Nos. 7, 97, 125, 


126, and 134). 


In 1972 the Congress appropriated funds for an expanded fire, textile, 
and fabric-flammability research program. ETIP was chosen to organize and 
monitor the effort. This effort moved in eanden with a push for promulgation 
of a flammability safety standard by the Consumer Product Safety Commission. 
In June 1974, ETIP contracted for research and development of a commercially 
practical process for rendering cotton/polyester apparel fabrics flame-resistant. 
Because of the complex nature of the problem it seemed unlikely that any single 
laboratory or company could succeed in developing an appropriate commercial 
solution within the two years deemed necessary for adequate and timely consumer 
protection. For this reason the novel approach of assembling a consortium of 
interested and experienced R&D organizations was employed. 

The consortium, headed by Clemson University, included participation by 


the University of Maryland, Research Triangle Institute (RTI), the Polytechnic 
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Institute of New York, the Southern Regional Research Center of the U.S. Depart- 
ment of Agriculture, the Hooker Chemicals and Plastics Corporation, the American 
Enka Company, and United Merchants and Manufacturers, Inc. The Dow-Corning 
Corporation joined the consortium in May 1975. 

The project had two objectives, product development and evaluation of "the 
feasibility of stimulating commercially important technological innovation 
through the organization and Federal subsidization of a consortium of academic, 
industrial, governmental, and private non-profit research laboratories."! 
Evaluation was to be based on the degree of success or failure in achieving the 
technical objective of product development. 

Evaluation of the project was done by Richard A. Rettig of The Rand 
Corporation under Project Nos. 97 and 134. Rettig assessed the consortium’s 
performance in terms of research results, commercialization results, and the 
lessons learned about consortia as an R&D management approach (to be discussed 
in Chapter aye 

Research: Basic knowledge about pe ieererieorton blends and flammability 
was advanced by good quality applied research that developed data which helped 
the industries involved make R&D decisions. Differences of opinion over these 
results were found among consortium participants and observers, however. For 
example, some felt the results replicated previous results, others that several 
significant areas had been missed, and others that the research was not well 
tied to previous research (pp. 20, 21). 

1. "Development of Flame Resistant Cotton/Polyester Fabrics," A Proposal 
Submitted to the U.S. Department of Commerce by Clemson University, Department 
of Textiles, May 16, 1974, p. 4. 

2. "The Use of Consortia for the Management of Research and Development: 


The Clemson University ETIP Flammability Prcject," Richard A. Rettig, The Rand 
Corporation, December 1976. 
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Commercialization: "The project did not result in a commercial system for 
flame-retarding polyester/cotton blends. On this there is no disagreement.. 
One view, advanced by the consortium leadership, is that several candidate 
systems of potential commercial value have been identified and deserve further 
exploration by the private sector. The polar extreme is that no new systems 
were identified except one presented at a Clemson symposium by the Sandoz 
Corporation, a finish developed independently of the consortium and not yet 
fully evaluated" (p. 19). 

Rettig pointed out, however, that between these two extremes a wide range 
of other opinions was held, including that industry never did expect commercial- 
ization and that the true worth of the project was "cross-fertilization" that 
will have an indirect impact on future product development for commercial 
markets (pp. 19, 20). 

A major factor impeding development of a commercial product throughout 
the project’s history had been uncertainty as to whether the CPSC would in fact 
promulgate a flammability standard and aie such a standard would require. 

Management: Rettig lauded the way the consortium had been organized, 
managed, and supervised. He pointed out, for example, that without continued 
government (ETIP) support consortium members "would not have been prepared to 
participate in its activities, primarily for antitrust reasons...(p- 5). 

While some participants expressed a preference for a stronger government role 
in technical supervision, most were of the view that the light (governmental) 
hand on research enabled that work to be determined within the consortium and 


facilitated greater learning about the consortium pattern of organization" 


(De Ost) 
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In general, Rettig found, "The consortium participants, and most of the 
non-participating observers, expressed the general opinion that the consortium 
was well managed, a sound concept, and deserved serious consideration as an 
instrument for confronting other problems shared by government and industry" 
(p- 21). He also listed several guidelines for future improvements in 


consortium-building. These will be considered in Chapter IV. 


ae Marketplace and Government Results 


ETIP’s R&D Program Area efforts succeeded in bringing IUS technology 
into the marketplace, if indications in 1982 were correct that actual construc- 
tion of such facilities would proceed at the targeted institutions. Further, 
although the consortium approach failed actually to result in a new technology 
for fabric flame-retardants, it appeared to have advanced the state of the art, 
demonstrated the viability of the consortium approach, and resulted in guidelines 
for similar future attempts. 

Perhaps more importantly, guidelines generated by ETIP for use in struc-— 
turing and/or approving budgets for demonstration grants (as a result of its 
studies of Federally funded demonstrations, to be discussed in the next chapter) 
were adopted by EPA, ERDA, and OMB. These changes in government agency percep-— 
tions and practices resulted directly from ETIP’s collaborative efforts with 
these agencies. The guidelines are listed in Chapter IV of this report and 
can be reviewed in greater detail in the final report by the Rand Corporation, 
"Analysis of Federally Funded Demonstration Projects," April 1976. (See 


footnote, 1 pe IV—26.) 
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Ee CONCLUSIONS 


This chapter has discussed a wide range of ETIP activities and the 


"products" they generated and disseminated -- whether those products were 


"things" or processes. The intent has been to highlight the impacts of 


these ETIP activities from each of two perspectives: that of the private— 


sector market-place and that of government line agencies. 


is Marketplace Perspective: 


Few technological innovations were diffused into the general economy as 


a direct result of ETIP experimentaion. 


oO 


In the regulatory Area only the FRA/ICC experiment was found to 
have directly stimulated diffusion of advanced technologies in 
the private sector, but even there the long-term effects of 
deregulation on the railroad industry remained unclear. 


ETIP had almost all of its few successes in stimulating technologi- 
cal innovation in its Procurement Area activities, though again 
actual diffusion of innovations into the marketplace was limited. 
Life Cycle Costing, Value Incentive Clauses and the use of perfor 
mance specifications did result in substantial savings to govern- 
ment purchasers, however, and.in some cases the innovative products 
purchased were commercialized. Only one minor product innovation, 
on the other hand, resulted from the long effort to introduce the 
VA to innovative purchasing techniques. 


The direct marketplace impacts of ETIP’s Economic Assistance Area 
activities were admittedly negligible, if they existed at all, 
unless one included those inventions brought to market by entre- 
preneurs aided by CPDC, itself operating in part with ETIP funds, 
but it should be noted that ETIP’s role in that project was 
limited to monitoring CPDC’s activities and developing a set of 
guidelines for future, similar endeavors. 


The "product" most likely to find general use as the direct result 
of ETIP’s intervention in R&D is the Integrated Utility System 
(IUS). By the closure of ETIP in late 1982, two universities and 
the City of Burlington, Vermont, were preparing to construct IUS 
facilities and a number of entrepreneurs were marketing the concept. 
Additionally, if flame-retardant fabrics someday find their way 
into the marketplace, it may well be at least an indirect result 

of ETIP’s innovative work in establishing a consortium to bring 
such technology closer to commercial development. 
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It should be repeated, however, that although ETIP was frequently judged on 
such a scorecard, diffusion of technologically new products into the marketplace 
never had been its primary objective. Rather as Rettig pointed out in his dis- 
cussion of the consortium to develop fabric flame-retardants, the proper role of 
government becomes the central question when commercialization is made the yard- 
stick:! 


(O)ne should be clear that there are two conflicting images of the 
public interest in commercialization. One image is of the "baton" of 
R&D results generated by Federal government-sponsored research being 
passed, as it were, to the private sector for rapid exploitation. The 
contrasting image is of the baton being carried up to the fence demar- 
cating the limits of government responsibility and being dropped on 
the other side. In the light of this second image, the non-exploita-— 
tion or slow exploitation of results by the private sector need not be 
regarded as a failure but can equally be understood as informed 
decision-making. 


It is more appropriate to guage ETIP b’ its achievements as an agent seek- 
ing to promote process changes within governmental line agencies that are likely 


to spur or, at the least, alleviate adverse effects upon innovation. 


268 Government Agency Perspective 


As an agent seeking to awaken government agency sensitivity toward techno- 
logical innovation and to alter government procedures and policies which affect 
it, ETIP’s record is considerably brighter. 


o In the Regulatory Area it (a) achieved deregulation of the public 
message communications industry by the FCC, a step which resulted 
at least to a limited degree in the introduction of competition 
into what had hitherto been a monopoly; (b) achieved deregulation 
of refrigerated and intermodal rail transport by the ICC, heighten- 
ing that agency’s appreciation of the need for stakeholder partici- 
pation and economic impact analysis in its processes; and (c) 
institutionalized ETIP perspectives at FDA, EPA, FCC, and State 
utility commissions that may eventually impact the pace and direc— 
tion of innovativeness by industry. 


l. Ibid., pe 29. 
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In the Economic Assistance Area it designed and institutionalized 
a system for monitoring the economic impacts of regulations on the 
venture capital market. Additionally, as result an of ETIP study, 
SBA altered its policies to help small business meet regulatory 
requirements. 


In the Research and Development Program Area it secured adoption 
of its demonstration grant funding guidelines by EPA, OMB, and ERDA. 


IV. ACHIEVEMENTS -- ETIP AS STUDENT OF CHANGE 


ETIP was created because it was discovered in the early 1970s that almost 
nothing was known about how government policies and activities affect the devel- 
opment and diffusion of new technologies. Its express purpose was to fill that 
knowledge gap by learning how to alter government agency activities so that they 
might foster technological innovation. In the course of its ten years ETIP made 
important advances toward that knowledge-building goal. 

Although the strategy of administrative experimentation was intentionally 
problem— and agency-specific and therefore not designed to develop generalizable 
answers, the repeated experiences of ETIP did in fact lead to some anwers which 
appear applicable across the several governnent activities considered. Others, 
however, address themselves exclusively to the individual program areas. This 


chapter considers both sets of answers. 


A. GENERALIZABLE LESSONS 


The overriding generalizable lesson ETIP learned was that the interplay of 
government and industry is far more complex than was believed in 1972. Many more 
factors are involved than merely a two-sided equation of government spur and 
industrial horse, factors which derive from marketplace forces which, given a 
free enterprise system, government actions can at best only modestly influence. 
If government is to have any significant impact on the course of technological 
innovation and its diffusion in that marketplace, it must be continuously sensi- 


tive to those marketplace forces and be willing to experiment continuously 
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with its policies and procedures so they might be designed to work with them. 
In effect, government must recognize it can do mich less as a spur that merely 
seeks to jolt the economy into action than it can if it adopts instead the role 
of one more horse harnessed to that economy alongside developer, producer, and 
user groups. 

Under this umbrella lesson, obvious once learned but far from obvious at 
ETIP’s beginning, the following more specific lessons were derived about govern- 


ment’s proper role in influencing technology innovation. 


l. Long-Term Commitment to Experimentation 

Because of the complexities involved, considerable time, money, and care 
need to be spent in experimenting with goverament policies and procedures if 
those policies and procedures are to be modified in a way that is both valuable 
and credible to government and industry decision-makers. Effective policy 
changes must be based on far more intricate field-data gathering and analyses 
than is usually believed necessary. Any agency which seeks to promote change 
in government policies and procedures, whether a third-party agency like ETIP 
or the line agency itself, must make a major commitment to this effort. 
This was noted particularly in the ETIP-FDA experimentation in post-marketing 


drug surveillance, but was also true across the breadth of ETIP—facilitated 


experiments. 


2. Stakeholder Collaboration 
ET[P’s strategy of involving all relevant stakeholders in each experiment 
from the outset proved invaluable in overcoming what would otherwise have most 


likely proven to be resistance by one or more parties, be they line-agency 
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personnel entrenched in traditional procedures and unwilling to risk failure 
(a risk implicit in any change) and therefore the disfavor of superiurs, or 
industry decision-makers accustomed to viewing government intervention with 
suspicion. 

It was learned (for example, in the experiments to deregulate rail trans- 
port and public message services) that extensive and careful studies conducted 
collaboratively with all private and public parties at stake can spark govern- 
ment agency actions that will foster technological change. 

Additionally, it was found that if these preliminary studies are guided 
by recognized experts in the substantive area being considered for experimenta- 
tion, and if those experts understand and appreciate the role of change agent, 
the studies are most likely to succeed in breaking down traditional stakeholder 
barriers by building credibility for the experimentation. In turn, the result- 
ing credibility and sensitivity to the need for change bodes well for the exper- 
iment itself and for the likelihood that the results of the experiment will sub- 
sequently be incorporated within Poverament agency policies and/or procedures. 

Once experimentation is decided upon, based on preliminary expert studies 
and field interviews with all stakeholders, those stakeholders should be involved 
continuously in a consensus group, thus assuring continuing feedback as to the 
effects of the experimental change. This strategy was followed successfully in 
a wide array of experiments. It afforded flexibility, mid-course "fine-tuning," 
and results both tangible and credible to stakeholders. In effect, stakeholders 
became partners in the change in order to achieve each his own objectives 
(sometimes but not necessarily divergent) rather than perceiving themselves as 


victims of the changes being wrought. 
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36 Interviewing for Expectations 


At the outset of each experiment with government change experimenters 
are best advised to interview all stakeholders, public and private, to determine 
their expectations. Arraying these expectations helped ETIP clarify the sets 
of objectives being sought from its experiments and determine whether those 
objectives were in fact realizable in the field. Continuous evaluation of the 
experiment by similar methods assures that those objectives continue to be 
realizable or, if they are found not to be, allow for sensible, data-based ~ 


mid-course corrections. 


4e Protection of Proprietary Rights 


Development and diffusion of new technologies necessarily involves new-— 
product patents which developers of those products wish to protect in order to 
assure their profitability. Too often private-sector developers distrust govern- 
ment intervention because of public disclosure laws that threaten these propri- 
etary rights. Mechanisms should be built into the experimentation -- whether 
in government procurement, in the gathering of feedback data from regulatory 
change, in the dispensing of financial or other government assistance to private- 
sector developers, or in the support of research and development efforts -—- that 
assures developers of new technologies that their right to profits from their 
endeavors will be protected. This was found to be a major factor in those ETIP 
projects which involved private-sector developers, for example in the experimen- 
tal procurement of low-noise lawnmowers which one manufacturer said he responded 


to precisely because his proprietary rights in the innovation were assured pro- 


tection. 
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De Top-Level Commitment 


Government agencies attempting to experiment with changes in policies and 
procedures should assure that top-level decision-makers within the agencies are 
committed to the experimentation to keep political and other pressures off 
agency experimenters and to avoid the disruptions all too likely in any effort 
that is as long-term as administrative experimentation must be. 

This lesson was repeatedly driven home in the FDA project to develop post-— 
marketing drug surveillance and in the FCC and ICC experiments with deregula- 
tion. It was also one of the hallmarks of the disappointing failure at experi- 
mentation with VA procurement that this lesson had not yet been adequately 


learned. 


6 Recognition of Agency Needs 


Line-agency personnel and decision-makers have varying needs from any 
given experiment with change. All these needs should be addressed in the design 
of the experiment with change if it is to result in tangible and credible policy 
and procedure modifications. 

Operational staff need both basic and advanced economic input from the 
experiment if it is to assess adequately the experiment’s impact on the 
private sector. Complex analysis models need to be developed and employed for 
evaluations of field data in order to provide decision-makers with valid policy 
recommendations. Those decision-makers, however, need simple, descriptive data 
on the economic impacts of current and proposed policy and procedure changes, 
rather than the elaborate analyses that undergird that data. 

The lack of sufficient and appropriate economic data was found even with- 


in such a financial communit y-oriented agency as SEC, traditionally a lawyer- 


dominated institution that emphasized the legal aspects of investor protection 
but gave little recognition to the economic benefits and costs of its actions. 
When it did use economic information-assessing tools, they were conventional 
and generally inappropriate to the needs. The ETIP-SEC project demonstrated 
that: 


o Conventional economic tools cannot be applied directly to policy 
analysis, and 


o Even when an agency collects considerable useful data, the data 
are typically not usable until both the collection and formation 
are tailored to the needs of the policy-analysis model. 

From these lessons it was derived that, because the data needed were not 
always available from the reporting forms collected by the agency, modification 
of those forms and collection of information from outside sources were both 
required. SEC implemented both recommendations. 

Additionally, sophisticated economic and computer-model training of agency 
staff was demonstrably needed if the analytical model was to be institutionalized, 
and that staff also had to develop the language skills to communicate the 
findings of their data-monitoring and evaluations to policy-makers in a simple, 


descriptive, and useful way if those evaluations were to have any impact on the 


policies being set. SEC also implemented these actions. 


Ve Real-Time Monitoring for Effects 


In experimenting with policies and procedures, government must monitor 
for the effects of its experiments during the course of the experiment. The 
value of this lesson was brought home to ETIP in project after project. For 


example: 
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e+e(R)eal-time monitoring...(of CPDC by Charleswater Associates, 
Inc.) was able to provide very specific details on CPDC activities, 
progress, and problems as these occurred. This not ony enabled ETIP 
to learn about CPDC but also undoubtedly made it much easier to de- 
velop the guidelines for CPDC operation which ETIP wanted. 
Retrospective analyses, while necessary on occasion, can never match the 
benefits to be derived from real-time monitoring. Such monitoring, however, 
makes it necessary that the design of the experiment include data components 
that can be collected as the effects of the experiment take place. Furthermore: 
---(T)he experimenter must try to keep faithful to recognized stan- 
dards of measurability so that at the end of the trial questions about 
cause and effect can be answered. An experiment should not be launched 
unless those involved are convinced that the process can be measured 
and that the data will be credible. Needless to say, such "proof" is 


an everyday question during the experiment as well as in the design 
stages which precede implementation. 


Be PROGRAM AREA LESSONS 


1s Regulation 


Government’s regulatory policies mid peoebaese. significantly affect pri- 
vate industry’s willingness to venture into the innovative technologies. This 
truth was borne out not only in ETIP’s interventions in government regulatory 
agencies but in procurement, economic assistance, and R&D experimentations and 
studies as well. Further, the significance of regulatory actions was found to 
be one set of complex interactions woven within the overall complexity of 


government-industry relationships. 


1. Project Overview, Project No. 78, August 7, 1981. 
2. Op. Git <3 pe II-6, Kelly, Ppe 167-168. 
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The complexity of the environment within which ETIP experiments with 
government regulation had to be carried out necessitated particularly heavy 
commitment of ETIP staff, as noted especially in the six-year experimentation 
with post-marketing surveillance of prescription drugs. 

Following are more specific and interrelated lessons derived from ETIP’s 
experimentation with government regulatory powers. (They are also derivative 


of more general lessons already discussed.) 


Market Mechanisms vs. Command/Control 

ETIP determined that a market-mechanism approach to regulation is both 
feasible and more desirable than the traditional command/control adversarial 
mechanism in which government adopts a standard of behavior and imposes it upon 
the private sector and is then forced to take on the role of policeman. Rather, 
by adopting market mechanisms that involve private-sector stakeholders, regula- 
tory agencies can enter into a collaborative process that will lead to more 
effective and realizable standards less likely to stymie industry innovative- 
ness. This was the strategy being successfully employed in the FDA experimenta-— 


tion with post-marketing drug surveillance at the closure of ETIP. 


Regulatory Doubt Can Impede Technological Advance 

While impending regulation may serve as an incentive for private industry 
to develop technologies to meet the proposed regulation, it also serves a 
forceful disincentive to the marketing of innovative products when there is 
doubt as to the direction, content, and timing of forthcoming regulations. 

In at least two ETIP projects development of new technology was in response 


to regulatory "push": the procurement experiment which resulted in government 
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purchase of prototype low-noise lawnmowers from a supplier who entered the com 
petition precisely because he saw a method of marketing a product that would 
meet forthcoming standards, and R&D experimentation with fabric flammability- 
retardants, which was focused directly upon an attempt to develop a product that 
would meet proposed safety regulations. However, in both cases doubts about the 
imminent regulations also operated counter to new-technology objectives. 
Regulatory experiments themselves were generally aimed more at deregulating 
industry so as to remove impediments to innovation (e-g., experiments with FRA/ 
ICC and FCC) and, as in the cases of FDA, EPA, and OSHA, eliminating the unnec- 
essary burdens of regulations. In each case these activities appeared to be 
factors which would improve innovativeness in the marketplace. Economic assis- 
tance experimentation (e.g-, with SEC and SBA) also had to consider regulatory 
impacts and were directed at alleviating the more deieterious of those impacts 
upon private-sector innovativeness. Whether in fact these government modifica- 
tions did foster innovations in the private sector required monitoring beyond 


the span of ETIP’s years. 


Technology Drives the Regulatory Game 


As witnessed in the project to deregulate public message services, techno- 
logy provides the impetus for regulatory action. The rapid rise of new communi- 
cations technology in this case forced the FCC to examine its policies and seek 
changes that would adequately reflect real-world events. 

Further, ETIP determined that docket-bound agencies (like FCC), in which 
policies are made as issues erupt or are put before them, would be better served 
by an ongoing capability to monitor and assess trends in the industries to be 


regulated so as to anticipate regulatory policy needs. 
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2. procurements 

Evaluations of its procurement experiments by ETIP contractors (SRI 
International and Research Triangle Institute (RTI)) generated a number of lessons 
about the ways in which government’s purchasing power can influence the rate and 
direction of private-sector innovation. 

In general, ETIP concluded that: 

o The use of LCC in Federal procurement can induce incremental tech- 
nology improvements in product design as well as substantial 
savings in energy and costs, as demonstrated in FSS purchases of 
energy-saving appliances; 

o The use of performance rather than design specifications in Federal 
procurement can induce technological innovation among potential 
suppliers as demonstrated in the procurement of low-noise lawn- 
mowers; and 

o The use of VIC in Federal procurement contracts did not result in 
widespread use of the clause by suppliers or in subsequent innova- 
tion, though it could be argued that this was due to provision of 
insufficient incentives. 

Mcre specifically, ETIP contractors identified the following elements to 


be considered in any attempt to employ government buying power to influence 


private-sector innovative behavior. 


Product and Industry Development Stage 

The single most important factor both SRI and RTI found to be influencing 
whether procurement strategies are likely to affect technological advance was 
the stage of maturity of the product and of its industry. They discovered that 
the use of procurement strategies appears most promising when the objective is 

1. This section is derived largely from Op. Cit., footnote 1, p. III-27, 
and from "Commercial Impact Evaluation of ETIP Procurement Experiments, Phase I 
and II,"" and "The Impacts of Federal Procurement on Industrial Innovation and 


Agency Purchasing Effectiveness: A Framework for Measurement and a Synthesis 
of Knowledge," by SRI International, both dated March 19, 1979. 
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to stimulate major innovations in the early stages of development. SRI reported 
that: 
The stage of product/industry development sets the context within 
which procurement incentives must operate.... (T)he opportunities 
for greatest impact on technology and product characteristics occur 
when the industry or product is young, and when products are in the 
early stages of the product life cycle. 
e-e(At the stage of product innovation) when alternative designs 
are being evaluated (by industry), policy incentives, including 
procurement incentives, have the greatest chance of producing funda- 
mental changes, since once development goes beyond this stage, the 
design parameters become more rigid and the cost of making design 
changes becomes greater. 

In the case of FSS’s procurement of energy-conserving appliances, for 
example, SRI found that the products being bought were "mature" and that only 
slow and uncertain advances in their technology could be expected to result from 
procurement strategies. 

e+e(A) great many more parameters have been identified as being at 
work in (these) LCC experiments.... The number of confounding 
influences potentially at work in explaining a single LCC impact (on 
mature-product innovation) raises a significant challenge to experi- 
mental design.... . 


A procurement strategy more likely to succeed in such circumstances, it 


was learned, would be aimed at achieving incremental product changes. 


Government As Major Purchaser and End User 


Federal procurement’s influence on private sector technological advance 
can be expected to be most effective when government is itself the end user of 
a product being purchased, thereby reducing market uncertainties. In such 
cases, RTI identified three factors at work to improve a procurement experi- 
ment’s likelihood of succeeding: 


o As user, government understands its own requirements; 
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o As user, government can communicate those requirements directly 
and accurately to potential suppliers; and 


o Suppliers recognize that government’s presence in the market is 
long-term rather than the short-term presence reflected in such 
cases as prototype purchasing of products intended for general 
market distribution. 

Government As Middleman or Broker 

The contractors found, however, that when government is not the primary 
consumer the opposite circumstances operate and the resulting procurement con- 
tracts usually have features that act as disincentives to technological innova- 
tion. Chief among these are provisions for termination at the convenience of 
the government and unilateral changes in the scope of the work, leading to 
marketplace uncertainties. 

The large share of risk shouldered by the developer of a new technology 
is often too large to add the non-market risks of government involvement. In 
these conditions, RTI concluded, some form of assistance by government to the 
private developer is often appropriate. Such cooperative agreements would allow 
"both parties...to contribute toward the desired goal, namely development 


and diffusion of a particular technology." 


Sensitivity to Commercial Market Strategies and Perceptions 


Procurement strategies aimed at stimulating industrial innovation can 
be effective when they are congruent with commercial market strategies. Both 
sensitivity to the nongovernment marketplace and techniques for effective 
communication with industry need to be developed, SRI concluded. Procurement 
strategies will succeed in influencing industry’s decision-making only insofar 


as they are perceived to alter the developer’s profits by: 
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o Increasing revenues from the firm’s innovative activities; 
o Lowering the costs of innovative activity; and/or 
o Lowering the risk associated with innovative activity. 

Additionally, government procurement strategies should recognize that 
the changes private industry is able to make in its products are limited to 
available technological knowledge unless the industry undertakes major restruc- 
turing of its R&D activities. This will severely limit a private developer’s 
interest in responding to a procurement that seeks a technologically advanced 
product. 

Government procurement agencies also should recognize that their objec- 
tives in procurement experimentation are often divergent from industry objec- 
tives. The most important determinants of operational appropriateness and 
feasibility of most procurement experiments to those within government gener- 
ally appear to be measurement/testing and technological trade-offs, while 
product/industry stages of development, nongovernment demand, minimum economic 
data, and government regulation appear to de the factors most important to 
industry in any new-technology procurement exercise. 

The disappointing experiment to apply VIC to FSS procurement was a case 
in point that illustrated the results of incongruence between government and 
supplier orientations. The experience demonstrated that publicity and training 
alone are apparently not enough to induce increased use of VIC because the large- 
scale effort to communicate VIC provisions to FSS personnel and suppliers did 
not result in many VCP’s from suppliers on subsequent contracts: 

o Innovative products were more likely to be bid on the contract 


in the first place, rather than brought forth during the term of 
the contract; 
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o Contracts tended to be too short for innovations to take place; 


o The cost of preparing VCP’s was to be shouldered by the supplier, 
thereby adding cost and risk; and 


o Suppliers were concerned that their innovations would set the stan- 
dard for the next procurement, thereby revealing a supplier’s 
innovative product and reducing competitive advantage. 

(It could be argued, however, that the relative failure of experimentation 


with VIC to arouse a significant level of response was due to its failure to 


offer sufficient rewards for development of innovative, cost-saving products.) 


Procurement Used in Tandem with Other Policy Tools 


When used in conjunction with other Federal government policy instruments, 
notably its regulatory powers, procurement experiments aimed at achieving tech- 
nological advance were found to have improved chances of success. 

This was particularly noted in the effort to purchase low-noise and safety- 
designed power lawnmowers, in which the procurement effort was intended to sup- 
port regulatory trends. (However, it should be noted again that regulatory lag 


and uncertainties could just as easily have frustrated the experiment.) 


User Roles in Development and Use of LCC Models 


A different set of lessons were derived from ETIP’s experiment with the 
Public Buildings Service (PBS) to design a comprehensive LCC approach to the 
analysis of options to meet Federal space requirements. 

Several factors contributed to the success of the project, however limited: 


o The model was developed within PBS and thus the owner of the new 
approach was in control; 


o The project leader in PBS was dedicated to model development; 
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o The computerized model significantly reduced calculation times for 
PBS space requirements studies; and 


o Government-wide policies supported in general the ideas of the LCC 
model and its use in PBS. 


Its institutionalization was hindered, however, by other factors, including 
failure adequately to involve important stakeholder groups: 


o The model was developed by a staff group rather than the real users 
in the appropriate line group; 


o Users were not directed by superiors or attracted by the project 
leader into a role of close support for project development; 


o The model was based on data not available at the time in PBS; 

o The model attempted to rationalize and influence a process which 
also involved political, economic, and design issues not wholly 
related to LCC considerations, thus limiting the model’s ability 
to influence the process; 

o Users were not trained to operate the model; and 

o The model was not fully supported by top management in PBS, which 
was still examining how to apply LCC to PBS policies at the close 
of the project. 

3. Economic Assistance 

In addition to those studies and monitoring efforts already discussed in 
Chapter III, ETIP’s Economic Assistance Area also conducted three efforts to 
develop knowledge about the role government does and can play in influencing 
the pace and direction of private-sector technology by direct infusion of 
economic aid and/or by ameliorating the environment to improve the chances of 
private-sector infusion of such funds: 

o A case study in which a range of government policies affecting 
a single industry (semiconductors) were examined to determine 
the relative effects and importance of different policy mechanisms 
and to assess the dynamics of policy impacts such as the timing 


of a policy’s implementation relative to the stage of the given 
industry’s development (Project No. 167); 
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o An examination of the impact of Federal subsidies on seven indus- 
tries (Project Noe 93); and 


o An assessment of various options available to government (including 
support for new private-sector technological initiatives) to 
determine which mechanisms could best deal with major socio-—- 
economic disruptions in commodity supplies caused by international 
cartel-induced shortfalls (Project Nos. 15, 17, 130A, and 132). 


The lessons derived from each of these, along with those derived from 


Economic Assistance projects discussed in Chapter III, are summarized below. 


Industry Strategies and Government Policies 

In Project No. 167 Charles River Associates, Inc. (CRA) examined the 
history of the semiconductor industry from its infancy in the 1950s when there 
was extensive interaction between government agencies and the industry because 
of government’s interest in space and military applications and a consequent 
willingness to fund R&D, through the industry’s early maturation in the 1960s 
when government demand decreased rapidly relative to total demand and resulted 
in a more laissez faire attitude by government, to the industry’s full maturity 
in the late 1960s and in the 1970s when ghverntent tax and trade policies played 
an increasingly important role and led to increasing frustration by industry 
executives. 

CRA drew these conclusions about the behavior pattern of the semiconductor 


industry and, by implication, of other high technology-oriented innovative 


industries: 


O eee(M)ajor innovations do not necessarily cost large amounts 
of money, and much R&D spending in the semiconductor industry goes 
for incremental innovation (p. S-3). 


1. "Innovation, Competition, and Government Policy in the Semiconductor 
Industry, Final Report," Charles River Associates, Ince, March 1980 (CRA 
Report #430.08). 
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Top management involvement (in innovation) and a high overall 
level of risk-taking are closely associated with major innova- 
tions. In contrast, capital availability and R&D spending are 
more closely associated with incremental innovation (p. S-3). 


---(S)trategic difference among firms...led to vigorous com- 
petition (the nature of which) changed over time. Typically, at 
the beginning of a new technology race firms placed strategic 
emphasis on major product innovation. In many cases this created 
opportunities for new innovative firms to enter the industry. Over 
time, as products became more standardized and sales volume in- 
creased, additional firms joined the race...(and) early parti- 
cipants sometimes left either because of innovation failures or 
because of increasing price competition (pp. S-3,4). 


---(The strategy group) easiest to enter, specialized products 
with incremental innovation, exhibited the worst financial per- 
formance. The major innovation groups showed the superior 
financial performance....- Good industry performance is found to 
result from having a diversity of strategies. These include major 
innovation, incremental innovation, second sourcing, and aggressive 
pricing (pp- S-4,5). 


From its examination of the role various government policies had upon this 


behavior of the semiconductor industry, CRA drew these conclusions about policy 


options: 


Government policy can have a-direct effect on firms pursuing incre- 
mental innovation strategies due to the importance of capital avail- 
ability and R&D spending...(which) are within direct influence of 

the government through its tax policies and interest rates (p. S-7). 


eee(Government influence on) firms pursuing major innovation stra- 
tegies is much more indirect because the important variables in- 
fluencing major innovation are top management background and commit- 
ment and a high degree of risk-taking. The major effect must come 
through policies that encourage people with the right talents and 
inclination to reach the top spots in firms that face the opportu- 
nities and have the organization to achieve major innovations 

(pp- S-7,8)- 


eee(Policies which could have such indirect effects include) 
capital gains or stock-option incentives, minimal legal restric- 
tions on personnel mobility, and low patent barriers (p- S-8). 


++e(F)ederal policy should attempt to put U.S. firms on equal 
footing with foreign firms in terms of capital availability and 
R&D spending in order to promote good international performance 
(p. S-8). 
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Oo .eee(A)n industry can be innovative without strong patent incen- 
tives (p- S-8). 


o Basic research appears to be important for innovation in the birth 
and childhood of an industry. However, as the industry matures, 
firms that do relatively little basic research can produce major 
innovations (p. S-8). 

o The relatively laissez-faire policy of the U.S. government (toward 
the semiconductor industry) in the last two decades has done a 
good job of encouraging major innovation, primarily through making 
it financially attractive for individuals and keeping legal bar- 
riers to entry and personnel mobility low (pp. S-8,9). 

O .eee(P)olicies such as capital gains taxation need to permit a 
reasonable chance of striking it rich; competition will take care 
of the possibility that too much of the benefits of innovation end 
up in the pockets of the innovators (p. S-9). 

Oo .-ee(I)ndirect effects of (tax and trade) policies can be as 


important...to innovative behavior as the direct effects (of 
procurement and R&D funding) (p.- S-9). 


Impact of Subsidies on Technology 


Major Federal subsidy programs gave an estimated $95 billion to private- 
sector producers and consumers in FY 1975. Because of the size of this direct 
aid, ETIP initiated Project No. 93 and contracted with CRA to study the effects 
of subsidies on seven industries and industry segments, their suppliers, and 
customers. The seven industries were municipal wastewater treatment facilities, 
mass transit, air transport, maritime transport, nuclear energy, health facili- 
ties, and technical publishing. 

As with virtually all ETIP evaluations of government-industry relation- 
ships, CRA concluded that "the relationships among subsidies, capital formation, 
and technological change were closely intertwined with a larger set of economic 


relationships," chief among them government’s regulatory policies which, in the 
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case of wastewater treatment facilities, "probably impeded technology change.""! 


The basic conclusion of the study: 

Subsidies have had their most significant impact on technological 

change where they were a relatively important element in a set of 

coordinated government programs which had as their specific goal 

the development and commercial introduction of new technology (p- ix). 

The subsidies studied, CRA found, did alter the quantity and quality of 
industrial output and in some cases also altered the mix of capital and labor in 
production. These changes in turn changed the rate and type of capital forma- 
tion and technological change in the industries and their suppliers in vary- 
ing degrees, and yet "no strong, specific relationship was observed between the 
subsidies and the rate or direction of technological change" (p. ix). 

That is, even if Pesarge subsidy succeeded in improving industry perfor- 
mance and its ability to raise capital, it was not enough to cause a discernible 
increase in the rate of Cechnbvop ical innovation. 

Another major finding by CRA: with only two exceptions "policy-makers 
apparently paid little attention to the possible impact of the subsidy programs 
on technological change a to the possible use of technological change to attain 
the goals of the subsidy program." As a result, "Policy-makers have missed 
opportunities to use technological change to help achieve the goals of the 
subsidy programs more effectively or efficiently" (p. ix). Citing the waste- 
water treatment facilities subsidy program as an example, CRA listed these 
shortcomings: 

o The short period of time...didn’t allow..-newly developed tech- 
nologies to be demonstrated, accepted, and incorporated into plant 
designs. 

1. "Subsidies, Capital Formation, and Technological Change: Summary and 


Conclusions," Volume 8, Charles River Associates, Inc., May 1978 (CRA Report 
302. 19) Pe viii. 
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o Massive short-term Federal commitment diverted R&D funds away from 
long-term development of innovative products and processes. 


o Once the physical plant was in place the cost of using newer, more 


cost-effective technologies would be too high because that physical 
plant would have to be removed. 


o Eligibility for certain subsidies "precluded a number of potential- 


ly innovative organizations from Federal economic assistance" 
(peexys 
To counter these shortcomings and failures of foresight, CRA developed 
a set of guidelines for policy-makers to help them determine whether give famaet 
intervention is warranted in a given situation, at what stage of industrial 
activity the intervention should occur, and what form the intervention should 


take. The guidelines cover: 


o Diagnosis of the problem, ranking of objectives, and analysis of 
the causes of the problem; 


o Designing a subsidy program including selection between demand-— 
side and supply-side subsidization; 


o Distributing the costs of the subsidy between public and private 
sectors; 


o Infrasructure requirements; 
o Consideration of effects on industry and the marketplace; 
o Matching the subsidy to the impediment; 


o The relationship between the recipient industry and its supplying 
industries; 


o Use of technological change as a policy instrument; 


o The interaction between the subsidy and government regulation of 
the recipient industry; 


o Experimentation with the subsidy to evaluate costs and benefits; 
and 


o Monitoring of the subsidy program for its effects. 
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These guidelines can be found in the report cited in the footnote on page 


IV-1 9 e 


Policy Options for Commodity Supplies 


In the 1970s the formation of international cartels to manipulate the 
flow and pricing of commodities (e.g., petroleum and bauxite) caused alarm that 
the U.S. economy might be crippled by shortfalls in important raw materials. It 
was believed that in some cases these shortfalls might be alleviated at least in 
part by technologically innovative remedies, such as development of synthetic 
fuels. Thus, the overall intent of ETIP’s Commodity Supplies projects (Nos. 15, 
17, 130A, and 132) was to determine what Federal policies were needed to deal with 
major socio-economic mieetotinna in those supplies where technological changes 
appeared to be an attractive remedy. 

Charles River Metoctates (CEA) explored how various government strategies 
might protect against disruptions in the markets for seven minerals: alumnimum/ 
bauxite, chromium, manganese, cobalt, platinum/palladium, petroleum, and copper. 
CRA concluded that high thipdeCudeveddencé; concentration of commodity supplies, 
or the existence of commodity cartels did not necessarily imply high risks for 
the U.S. CRA also found that:! 


o With the exception of chromite, stockpiling was not an appropriate 
government response to critical materials; 


o Some amount of stockpiling might be useful as minimal protection 
against short-term cut-offs, but stockpiles in private industry 
and/or government depots already met the need; 


1. Derived from "Policy Implications of Producer Country Supply Restric- 
tions," (nine volumes), February 1977, and "Commodity Supply Policies," August 
1977, both by Charles River Associates, Inc. (NBS-GCR-ETIP 77-36 and NBS-GCR-ETIP 
77-38 respectively). 
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o Tarriffs, quotas, or subsidies were not desirable policies to 
reduce import dependence; they would create unnecessary costs 
to the U.S.; 

o Existing technology, which might provide a means of substitution 
for disrupted supplies, was an inefficient approach for the govern- 
ment to support, but R&D aimed at producing substitution technolo- 
gies more in line with projected mineral costs might be an appro- 
priate government effort; 


o Other policies might help alleviate the potential supply problems, 
such as encouraging conservation of minerals and ocean mining; and 


o Technical knowledge about substitute materials that might be used 
and other technical information that would be of value to users — 
of raw materials is widely scattered. Information diffusion chan- 
nels are inefficient, fragmentary, and sometimes inaccurate. As a 
result consumers of raw materials cannot effectively plan for 
potential sudden shortages and cannot participate intelligently in 
the development of Technical alternatives. 

As a principal spin-off product of this report, CRA developed a computer- 
ized analytical framework for future critical material applications and for 


evaluating policy options, including supply- and demand-side technological 


options, stockpiles, tariffs, and subsidies. 


Small Businesses” Special Problems 


Technology-based small firms face unique problems in attempts to meet 
Federal regulations and remain financially viable. These include.a lack of an 
effective "voice" in regulatory agencies and legislatures such as larger compa- 
nies enjoy through trade associations and other vehicles; a lack of expertise 
in or access to and gudere canning of state-of-the-art technologies that could 
help them satisfy regulatory requirements; a traditonal concern for “end of the 
pipe" as opposed to "process" solutions to technological problems; difficulties 
in raising capital and realizing economies of scale; and fragmentation of regu- 


lations and regulatory agencies that affect small business. Because of ETIP’s 
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initiatives, SBA addressed these problems by adopting several ETIP-generated 
recommendations for advocacy on behalf of and technical assistance to small 


firms. 


4e Research and Development 


ETIP’s efforts to evaluate and intervene in Federal R&D and in demonstrations 
of R&D-spawned products were its only activities that appeared to have "the oppor- 
tunity to modify and experiment with government processes that can directly impact 
the process of technological innovation in the private sector"! (emphasis 
added). And yet, as revealed in virtually every study by ETIP of Federally 
funded R&D, government’s impact on innovation through such direct support has 
been greatly overestimated. 

In part because of the relatively high visibility of government’s supposed 
impact on innovation through R&D funding, and in part because such funding was 
initially intended to be at the core of ETIP’s mission, it was in this field 
that ETIP’s examination of traditional processes and attempts to experiment with 
those processes most directly led to a fuller comprehension of the complex 
interplay between private-sector innovativeness and government intervention. 

For this reason this section considers in some detail the "lessons learned" by 
ETIP about that interplay and how to improve its track record. 

As six case studies conducted by Arthur D. Little (ADL) in Project No. 19 

("Federal Funding of Civilian R&D") demonstrated, Federal R&D funding is never 


enough to stimulate technological change in the private sector because such 


le Op. Cite, Pe III-6, Kelly, Pe 90. 
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financial support is only one among many factors which contribute to creating 
a climate conducive to change: 


R&D cost is a small part of the total cost of bringing technological inno- 
vation into the marketplace. This fact is often overlooked by Federal 
policy-makers in both the executive and legislative branches. It is one 
of the reasons why many U.S. companies with proven records of developing 
and marketing new products often shun Federal R&D funds and why so many 
Federal R&D products are shelved. 


The "mythology" of Federal R&D is derived largely from the success stories 
of the Manhattan- and Apollo-type projects. What is too often overlooked 
is that in such instances the government was not only the funder of R&D, 
but it was also the customer for the resulting hardware and systems. 


Now the government’s role as customer is diminishing for a growing frac- 
tion of the R&D it funds. R&D targeted to the civilian sector has 
increased from 23% to 35% in the last six years. The new consumers are 
industry, local government, and private citizens. In these instances, the 
funder of R&D, the: performer of R&D, the manufacturer using R&D-generated 
knowledge, and the customers are separate and autonomous elements related 
through the workings of the market. 


Government intervention in the marketplace may be called for when 
deficiencies in its workings are detrimental to the economic or social 
well-being of the country, (but) (m)erely funding civilian R&D is 
insufficient to correct those situations. 

ADL concluded that "policies for Federal funding of civilian R&D should © 
be formulated in the larger context of the complex process of innovation." 
Weaknesses ADL found in government’s traditional attempts to support R&D 
intended for the commercial marketplace included the following: 

o Federal R&D policies have been mainly the result of decentralized 
decision-making by mission-oriented departments and agencies 
(p> 3). 

O eeeThe process of innovation, involving the private sector as 


the agent for changes, was poorly understood by the Federal R&D 
funding agencyeee. 


1. "Federal Funding of Civilian Research and Development, Volume 1: 
Summary," Arthur D. Little, Inc., February 1976 (NBS-GCR-ETIP 76-03) p. l. 
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o (That) process is often not a linear, point-to-point process, 
in which each step is part of a systematic progression. Innova- 
tion is more adventurous, more spontaneous, and more opportunistic. 
The triggers are many. Government incentives can be triggers, pro- 
vided that they are formulated with full knowledge of the private- 
sector environment...(p. 4). 

o There is rarely any Federal strategy aimed at propagation and 
application of research and development results in the industrial 
society-e Indeed...the rules of practice for industrial parti- 
cipation in mission-agency R&D (may) create built-in barriers to 
commercialization...(p. 5). 

O ee-(T)he ability of business to perform as the agent of techno- 
logical change is dependent upon a very wide range of factors and 
influences which interact among each other: timing, perceptions of 
the market, the calculus of opportunity and risk, the objectives of 
the firm, expectations of returns, the structure of competition, 
and to some degree, the role of government wearing its several hats 
as policy-maker, R&D funder, and regulator...(p. 6). 

ADL concluded that "Federal R&D funding policies must be formulated as one 
possible component in this complex framework of interacting factors which trig- 
ger the process of innovation, some of which are significantly influenced 
through government policies and actions other than R&D funding" (p. 6). High 
among these influencing actions are regulatory policies and procedures which, 
as ETIP learned from its Flammable Fabrics Consortium Project (No. 7), can add 
to the complexity and serve as a "confounding influence" when regulatory outcomes 
are uncertain. (From this experience ETIP identified a potential for increased 
interaction between R&D and regulatory activities.) 


‘The following are other lessons learned by ETIP in its experimentation with 


and evaluations of Federal R&D and demonstration programs. 


Technology Push vs. Market Pull 


Government’s principal shortcoming in its R&D activities was shown to be 


a failure to understand that technology was more likely to be "pulled" into the 
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marketplace by encouraging favorable conditions among potential users of the 
new technology than it was to be "pushed" into commercialization by government 
subsidization of technology producers. Stated another way, Federal incentives 
to create or aggregate a market for civilian technology are more appropriate 
than Federal funding of civilian-oriented R&D. 

The ADL study noted that when Federally funded R&D efforts resulted in 
commercialization it was almost always because policy measures undertaken by 
the company developing the innovative product either caused or stimulated 
market demand. That is, inventions were most often "pulled" into the market 
by use of appropriate incentives, not "pushed" by the R&D itself. 

From its investigation of 24 demonstration projects spanning a wide range 
of Federal agencies, technologies, and project characteristics (Project No. 11), 
Rand found that what was true for R&D was also true for Federal funding of 
demonstration projects. In its final report, Rand stated:1 

In the absence of a well-articulated market demand the pursuit of 
demonstration projects is an especially risky activity; whatever 
successes are achieved are accompanied by many failures. 

ETIP explained its "market pull" strategy in the background briefing paper 
prepared for Secretary of Commerce Elliot L. Richardson:2 

Civilian research and development programs have been viewed by many 
people as the government trying to find practical uses for the tech—- 
nology it had developed. This was viewed as creating a technology 
push program. ETIP’s view is that...(research can become) a tech- 
nology pull process...(if) the problem definition activity... 


involves more deeply the ultimate user of problem-solving products 
being developed. 


1. ‘An Analysis of Federally Funded Demonstration Projects: Final 
Report,'' Walter S. Baer, Leland Le Johnson, and Edward W. Merrow, April 1976 
(R-1926-DOC). 

26 "Experimental Technology Incentives Program: Background for a Briefing 
for Secretary of Commerce Elliot Le. Richardson," 1976, p. 494. 
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Based on 46 case studies, SRI International developed and tested a set of 
36 guidelines to help Federal R&D managers improve the commercialization of 
their results. The guidelines are distributed across stages of management from 
program planning and project selection to final project review and evaluation. 
They concern market planning and analysis, market intelligence and communication 
techniques, and selection and management of R&D performers. Of all the cate- 
gories, SRI found that market planning and analysis factors had the greatest 
importance during planning and project selection. Most projects, SRI found, 
failed to include market analyses in these early stages and therefore failed to 
find market acceptance. 
Our findings indicate that the most important recommendations for 
improving the management of Federal R&D fall into two major areas: 
(1) Market Planning and Analysis, and (2) Market Intelligence and 
Communications Techniques. The first emphasizes the importance of 
orientating Federal R&D agencies to the marketplace and of designing 
projects to meet producer and user requirements rather than promoting 
a technology per see The second stresses the importance of communi- 
cations and cooperation among the various parties -- Federal R&D 
funding agency, R&D performer, potential producers, potential users, 
and other governmental and nongovernmental bodies -- whose actions 
affect the transition of R&D to the marketplace. Both sets of acti- 
vities embody a philosophy of market responsiveness that current 
Federal practices make difficult to realize.! 
SRI displayed its guidelines in matrix form. As noted earlier, "Market 
Planning and Analysis" had the greatest array of activities recommended to be 
undertaken in the program planning and project initiation stages -—- further 


emphasizing the importance of "market pull" in the commercialization of innova- 


tions. 


1. "Management of Federal R&D for Commercialization," Final Report and 
Executive Summary, SRI International, September 1978 (NBS-GCR-ETIP 78-58 and 
78-57) Pe 2- 
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Limited Value of Demonstrations 
Federal funding of demonstration projects have limited effect on commer- 


cialization of R&D results. Such demonstrations, ETIP learned, had some market- 


place effect only when: 


Oo eee(T)he principal technological problems (involved) had been 
worked out beforehand; 


o They “involved non-Federal cost sharing, while those funded 
entirely by the Federal government resulted in little or no dif- 
fusion." 


o They resulted from initiatives by private companies or local 
public agencies rather than having been "pushed" directly by the 
Federal government; 


O eee"(T)here were obvious manufacturers and purchasers of the 
new technology, and markets for similar products existed"; and 


o They did not face "externally imposed time constraints." 
Based on these findings, Rand drew the following conclusions about the 
appropriateness of demonstration projects and their characteristics that con- 


tribute to success or failure at diffusion: 


o Demonstration projects have a narrow scope for effective use. 
They are most effective when diffusion is hampered by lack of know- 


ledge...-(by) potential adopters about the use of the technology 
under commercial operating conditions. But demonstrations are 
appropriate only when these uncertainties are not large and when 
there is a strong rationale for Federal involvement. 


o Demonstration projects appear to be weak tools for tackling insti- 


tutional and organizational barriers to diffusion. Other govern- 
ment interventions, such as changes in regulations or subsidies, 


may be more effective than demonstrations in stimulating diffusion 
in such situations. 


o Large demonstration projects with heavy Federal funding are parti- 
cularly prone to difficulty. Heavy Federal investment tends to make 
projects highly visible and vulnerable to political pressures 
detrimental to success. 


l. Op. Cit., IV-26, Baer, et. ale, pe v. 


oO 


oO 
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On-site project management is generally effective. Whatever 
management problems arise are overshadowed by (the need to address) 
other more serious problems noted above. 


Dissemination of information from demonstration projects is 


generally not a serious problem. When projects fail to...(be 
diffused commercially), they generally do so not because of weak- 


ness in the information network, but for other reasons noted above 
(p- v-vi). 


Among the more important guidelines identified by Rand: 


oO 


eee(T)he demonstration (should be conducted) on as small a scale 
and with as little initial visibility as possible. 


eee(L)arge projects (should not be emphasized) at the expense 
of small ones involving incremental improvements to existing 
products or processes. 


eee(P)olitical pressure to demonstrate before a technology is 
well in hand (should be resisted). 


e+e(E)nough time should be allowed in the project’s schedule 
for slippage, especially...(in) large projects with significant 
technological uncertainty (p.- vi). 


Rand recommended an initial exploratory study to answer the following 


questions: 


What is the rationale for Federal involvement in relation to on- 
going private-sector activity? 


In what specific ways will the project generate new knowledge or 
otherwise stimulate diffusion? 


Who are the principal decision-makers in determining whether the 
new technology is adopted, and how will they be influenced by the 
demonstration? 


What evidence is there of interest in the project among potential 
adopters or other target audience? (pe vi). 


In planning and implementing the project, Rand recommended that: 


oO 


fe) 


Potential adopters and other target audiences, including regula- 
tory agencies where relevant, help plan the demonstration; 


Concrete planning be done at the local operating level with Federal 
review, and not by the Federal agency; 
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o Demonstrations include cost-sharing by private-sector firms with 
incentives to diffuse the technology; 


o When demonstrations involve continuing experimentation, the distinc- 
tion between these activities be made clear to potential adopters; 
and 


o Because demonstrations involve trade-offs among costs, performance, 
and time, the time element -- rather than cost and/or performance -- 
should be relaxed (p. vi-vii). 
Other guidelines developed by Rand can be found in Rand’s final report, "An 


Analysis of Federally Funded Demonstration Projects" (see footnote, pe IV—26). 


Alternative R&D Strategies 

The ETIP/HEW project to spur IUS applications at universities and hospital 
complexes demonstrated that government does have alternatives to heavy inane 
investment in R&D and demonstration projects. In the case of IUS, the success-— 
ful alternative was provision of engineering and economic information rather 


than a massive infusion of funds. 


The implication of these findings is that under certain circumstances 
the government need not mount expensive demonstration projects but can 
leverage with rather modest amounts of funds fhe expenditures of 
capital funds by the users of the technology. 


The IUS project, however, did result in three operational "lessons": 


o The need for close supervision of and coordination with any 
cooperating agency to prevent, correct, or reconcile goal diver- 
gence during the conduct of the experiment; 


o The need for substantial support from upper management of the 
cooperating agency during the course of the experiment; and 


o The need for substantial support from upper management of the 
cooperating agency during the course of the experiment; and 


1. Op. Cit., footnote 2, p. IV-26, p. 497. That "users" of IUS were 
encountering difficulties in securing construction funds could not be 
attributed to a failure by ETIP; rather, ETIP "succeeded" in securing commit- 
ments from these users because, even though it used its funds only to demon- 


strate the feasibility and practicality of IUS, it still secured commitments 
to build. 
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o The need for continued flexibility in the course of developing 
experiments and dealing with impediments that arise in the course 
of conducting the experiments. 


Consortium Viability 

A consortium of individuals and institutions representing all interests 
in research and commercialization of a new technology can be an effective mech- 
anism for directing government research funds toward practical marketplace uses 
in certain circumstances, as ETIP learned in its experience with a consortium 
to develop flame-retardants for fabrics. The ultimate failure to introudce a 
commercial system for flame retardation was due not to intrinsic shortcomings 
of the consortium approach but to technological limitations. However, ETIP 
learned that use of consortia must recognize and address certain difficulties: 


o Substantial efforts are necessary if adequate communication among 
consortium members is to be developed and maintained; 


o Problems of protecting firms” proprietary laboratory equipment and 
practices from the full public disclosure requirements associated 
with government funding must be addressed; and 


o The antitrust implications of. consortia are paramount in the minds 
of private firms and must be countered. 


In his evaluation of the Clemson University eennart tames Rettig identi- 
fied these major factors as influencing the project: 


o Personal and professional characteristics of the project director, 
identified by many as “the critical factor(s)" in the consortium’s 
success; 


o Leadership located in a university department that had a strong 
interaction with the industrial community; 


o Institutional incentives to participate, derived from "positive 
benefits and the absence of negative costs" and from the fact that 
"private firms inside and outside the consortium were actually 
involved in cost-sharing with the project"; 


Tee Op. Cit. 4 P>s III-6, Kelly, pe 89. 
2e Op. Cltia, Pe ILI-47, Rettig. 


Rettig 
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Individual incentives to participate, derived from "preexisting 
channels of communication between the project director and the menm—- 
ber representatives..-which had previously contributed to 
familiarity, trust, and confidence"; and 


ee." (E)xtensive, if not extraordinary expenditures of time and 
energy for communication with consortium and non-consortium orga- 
nizations alike," which assured a steady flow of information vital 
to such collaborative efforts. 


also listed suggestions made by interviewed participants as to how 


the consortium might have been improved. The following are derived from those 


which appear most relevant to future attempts at R&D consortium-building; 


oO 


oO 


Rettig 


Oo 


Total resources should not be fragmented among consortium members. 


Considerable attention should be given to early involvement of all 
stakeholders, in this case to include commercial fiber and textile 
firms, prospective users, other researchers besides chemists, and 
an advisory committee of nonparticipants. 


A planning period should precede the research project itself. This 
would facilitate project build-up after the contract award and make 
for better use of subcontract research work. 


Redundancy of research work should be allowed at the initiation of 
the project for definitional purposes, with severity exercised in 
the termination of less promising lines of work. 


A public meeting should be held to report initial results. 


More emphasis should be placed upon generating underlying scienti- 
fic knowledge than on applied work. 


Efforts should be expended to integrate the results of the 
research with the appropriate body of scientific literature. 


added a few "balancing comments" to the suggestions above: 


No attempt had been made to discount the bias of the participants 
he interviewed. 


The utility of acting on any of the above suggestions should be 
set against the costs of doing so; and 


His report was completed without circulation to the consortium 
participants and thus did not have the benefit of any critical 
reaction they might provide. 
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Several general conclusions were drawn about the values and limitations of 


consortia as mechanisms for R&D management. The values: 


ie) 


Consortia may be attractive "where the resources...(to address a 
technical problem comprehensively) are not available in a single 
research organization and where the talents of several organiza- 
tions can be easily assembled." 


Consortia may offer a sensible way to offest limitations in the 
capabilities of the Federal government for detailed technical 
management of a project. 


"Consortia appear to be of high value where it is important to facil- 
itate the flow of scientific and technical communication across 
organizational boundaries. In the Clemson University project, 
university research groups became much more sensitive to the pro- 
blems of commercialization...(and industrial firms developed) 

greater appreciation of underlying factors in flammability, infor- 
mation bearing both on short- and long-term work." 


"Increased communication..-appears to be one of the chief values 

of the consortium approach." This was illustrated by the fact that 
private companies conveyed "more information...without threat to 
proprietary information than would be the case otherwise... 
(because of) personal knowledge of and trust in the individuals to 
whom such information (was) being transmitted" (p. 25). 


The limitations: 


Oo 


"Costs of coordination are high. These costs can be compensated 
for by energetic leadership committed to the task...(b)ut these 
costs should not be ignored or underestimated...." 


Consortia "may not facilitate collective participation in project 
management" when they are "of relatively short duration and geo- 
graphically dispersed" because temporary organizations place "a 
premium on developing consensus and minimizing conflict.... 

(This) may incline such organizations toward risk-adverse behavior 
in scientific terms and inhibit boldness in the exploration of 


technical possiblities." 


",.-(0)ne recurring problem will be the handling of proprietary 
information" (p. 26). 


Rettig concluded that "a thorough appraisal of consortia...would require 


a comparative analysis of this approach with several other more traditional 


approaches." 
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Importantly, Rettig found that without active government intervention by 
ETIP the consortium probably would never have been organized because of anti- 
trust fears on the part of participating industrial firms. Thus, the consor-—- 
tium approach seems to require government-supplied incentives. Rettig cau- 
tioned, however, that such government action appears attractive only if "there 
is a clear public policy rationale for government intervention in the first 
instance. The desire to commercialize the results of government~-sponsored 
research is not, a priori, sufficient policy basis for efforts to do so" 


(p- 29). 


Political Sensitivity 

A final caveat: ETIP discovered that experimenting with R&D involves 
"a relatively large commitment of resources...compared to the effort required 
to develop, say, a regulatory experiment given the current administration’s 
interest in regulation reform."! 

Throughout R&D experimentation and analyses ETIP witnessed the political 
sensitivity of Federal R&D funding issues. This included not only the potential 
for competition and rivalry among competing agencies and their theorists, "but 
also the reluctance on the part of those charged with implementing a political- 
ly popular Federal R&D program to experiment with that program (and, by implica- 
tion, srisk faitere ye ite 

Even in a climate favorable to R&D, then, in which Federal policy active- 
ly pursues R&D results (as during ETIP’s early years before de-emphasis in favor 
of regulatory experimentation), political considerations loom large for experi- 


mentation with R&D approaches. (This was also pointed out, as mentioned earlier 


l. Op. Cites pe IlI-6, Kelly, Pe 92. 
2 Ibid. 
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in this section, in the guidelines for demonstration projects developed for ETIP, 
in which the Rand Corporation urged that such projects be kept small and with 


low visibility in order to avoid "political pressures.") 


whétleyveb ‘eit Sp pre) Miwasaesh nenlahiey arin 


| Leoae “iF 


' <P) F 
aj i 
t 
é 
= ‘y 
' 
i 
‘ 
a< yf 
te ’ 
7 ‘ 
i 
r x ‘ 
ine 
os 
be , 
é 9 


&s 


a8 Ue Wigneterig: doo Sesto toa reskaneg2b Od 
ct aw. evs aed Saat ed tev" tics rset 


e7n0nt- mee . te ic ent ivees 
Lat iw poate ai rect ive ot 
t tsar ‘'necerunne fen: re: “ 
- Yeou' te a? poverywanith 
f) is 1.4 


“et a 9 he y athe 


Liat as ry.c ie * vorta 


uel wich . Sy eeeegs io2g<B 


= 


mach tit neces (dehy le 


Liv, in watch Vederal pabiggi 


y years before de-gaphani 
~ A : ' : : : 
. te — ‘oom 
a a 7 


Ve. CONCLUSIONS 


In 1979 a panel of outside observers evaluating ETIP for the Department 


of Commerce circulated a report! which stated in part: 
As in any complex learning venture, the early work of ETIP was some- 
times naive and clumsy, with false starts, wheel-spinning, and 
mistakes. They are inevitable elements in a learning process, and 
ETIP learned a great deal from both its successes and failures. 

Thus ETIP, as an experimental program, was as much a student of its own 
activities as it was of other agencies. That is, it strove to learn about 
government-to-government activities that might most efficaciously induce change, 
and continued to learn from both its successes and failures until its termina- 
tion. This chapter summarizes both ETIP’s weaknesses and its strengths as factors 
that contributed to those successes and failures -- its weaknesses and strengths 


as perceived by those most directly involved. It closes with a summation of the 


answers to the three basic questions ETIP asked. 


Ae WEAKNESSES 


The "weaknesses" derived more from the environment within which ETIP 
was required to experiment than from characteristics inherent in ETIP 
itself. Nor were the weaknesses by any means unanimously accepted as such by 
all participants and observers of the ETIP decade. Some could just as easily be 
considered as the necessary trade-offs of the more significant benefits to be 
derived from the very trademarks that gave rise to the "weaknesses." In either 
case, future experimenters with government policy would be wise to bear them in 


mind. 


1. "An Evaluative Report on the Center for Field Methods," Evaluation 
Panel draft report, May 25, 1979. 


ibe No Overall Strategy 


One of the most frequently cited flaws in the way ETIP carried out its 
mission was that it failed to develop an overall strategy which would tie incre- 
mental experiments together into a network to make the whole larger and more 
important than the sum of its parts. This resulted from ETIP’s piecemeal stra- 
tegy designed to achieve results of immediate practicality. 

Although this was most true of ETIP’s early years, mid-course corrections 
by ETIP in its later years were achieving their goal of coordinating an overall 
strategy. Had ETIP survived several years longer, it is possible this short-— 
coming would have been resolved, but this would have required a long-range con- 
mitment to systemization (as opposed to quick, popularly recognizable Bi. 
meal results) on the part of both legislative and executive branch political 


leaders, a commitment lacking in the early 1980’s. 


2e Organizational Tug-of—War 


The chief organizational difficulty that impinged upon ETIP’s chances for 
success throughout its history took the form of a tug-of-war between those who 
wanted elaborate and lengthy research protocols and those who wanted quick 
results. The first suspected the second of politics and advocacy, considered 
anathema to ETIP’s stance as neutral third-party facilitator-researcher, and the 
second disdained the former as "too academic." 

Overlaid upon that difficulty, and in part growing from it, were repeated 
changes in management structure and leadership, which frustrated attempts to 


maintain the continuity needed for a coherent overall strategy. 


3. The Pressure of Time 

The average lifespan of an ETIP project was two to three years, too short 
a period to assure thorough testing of hypotheses and mid-course corrections, 
development of basic knowledge, or marketplace diffusion of new technology. 

(A significant exception to this, and one which "proved" the point that 
significant results take time, was the six-year history of the FDA experiment.) 

Particularly in ETIP’s early years, the general strategy was to seek out 
and respond to overtures from those administrative agencies already eager for 
ETIP funds and/or expertise in order to improve their own internal efficiency or 
effectiveness. Such a strategy, while conducive to the production of "results" 
in the short-term deemed necessary for continued congressional support and 
funding, worked counter to those who saw need for the slower pace of rigorous 


application of scientific method. 


4e I11-Prepared Hypotheses 


One direct result of the time pressures, felt most in ETIP’s early years, 
was initiation of experiments to test ill-advised or incomplete hypotheses. 
This shortcoming was noted repeatedly. Hypotheses often were built on an 
incomplete understanding of the problems, a prime example being the hypothesis 
that post-marketing surveillance of drugs could provide a safety net for drug 
approval in such a way that early approval could be given and the climate for 
pharmaceutical innovation thereby improved. The FDA experiment demonstrated 
that changes in the drug-approval process, including industry-government col- 
laborative approaches, would also be necessary if the experiment was to 


succeed. 


aye Changes in Context 


Changes in the context within which ETIP experiments were carried out (by 
design in the "real world" subject as it is to the moving targets of economic, 
social, and political shifts) further frustrated attempts to test definitively 
ETIP’s originally stated hypotheses. 

While ETIP’s strategy of flexibility was adopted in recognition of these 
"real world" shifts, this flexibility did have negative effects as well as the 
beneficial ones already cited (e.g-, stakeholder involvement for consensus and 
commitment—building). The significant negative effect was further frustration 
for those eager to build ETIP progress slowly and scientifically in order to 
produce valid generalizable answers to basic questions posed about government’s 


impact on innovation in the private sector. 


6- The Uncertainties of Behavioral Sciences 

Generalizable knowledge about government’s impact on innovation is not only 
not widely available but also is not likely to yield to ETIP’s experimentation 
methods and constraints. This is true both at the level of the ETIP Program 
Area (e.g-, the impact of government procurement on innovation) and at the indi- 
vidual project level (e.g., the impact of life-cycle-based purchasing on 
appliance technology). 

The difficulty is one often encountered in studying human behavior and 
social systems. Both government program delivery and industrial innovation are 
activities which should be defined as social systems and social behavior, involv- 
ing as they do such subtle and varying factors as human unpredictability, 


signals, incentives, etc. Such fields of endeavor have rarely been susceptible 


to methods of scientific inquiry. 


Be. STRENGTHS 


Given these shortcomings, however, ETIP was remarkably successful in proving 
that the traditionally viewed "hidebound" bureaucracy can in fact be moved to 
change in an orderly fashion that minimizes risk and, more importantly, that the 
interaction between government and the private sector can itself be modified in 
a way that is credible and satisfactory both to those inside government and to 
those in the private sector. 

Final evaluation of ETIP’s program is properly a function of stakeholder 
viewpoints. The following is a selection of opinions about ETIP from such 
stakeholders -- those in government agencies who experienced firsthand the ETIP 
change process. It is by no means a scientific sampling, which would be beyond 
the scope of this final report, nor in any way representative of the spectrum of 
opinions about ETIP. Rather, it is an attempt to discover through these agency 
reactions what ETIP’s assets were perceived to be by those directly involved 


outside ETIP. 


ls. Agency Responses 


Food and Drug Adenine tae tele 


There is no doubt that this project has had a tremendous impact upon 
certain activities within the FDA. The project has clearly resulted 
in changes both within the agency as reflected by programs and 
actions, and also outside as reflected in some of the recommendations 
regarding PMS (post-marketing surveiliance) we have received.... 


1. The following is extracted from a letter by Jerome A. Halperin, Acting 
Director, Office of Drugs, National Center for Drugs and Biologics, U.S. Food and 
Drug Administration, to Robert Parsons, Director, Experimental Technology Incen- 
tives Program, National Bureau of Standards, July 15, 1982; and from "A Policy 
Experiment in the Regulation of Human Drugs," by Jerome A. Halperin, Drug Intelli- 
gence and Clinical Pharmacy, Vol. 14, pp- 845-847. 


The 18-month gestation period for the first contract proposal was not 
all a "peaches and cream" consensus-building exercise. The process 
was more like the mating dance or porcupines, stylized and ritualis- 
tic, slow and careful. We found we had basic problems in vocabulary; 
we spoke a different language. Words like "system" that seemed to 
have simple meanings, meant different things to the FDA and to the 
Eli Peisc 


(I1)t is essential that we state for the record that ETIP has been 

an effective agent of change. (Emphasis added). At the outset 

of the project we were told that ETIP was an organization interested 
in policy experiments with Federal agencies to determine if agencies 
could be provided with incentives to change their policies in ways 
to stimulate innovation. I believe without question that this goal 
has been met by ETIP in this project.... ETIP provided the incen- 
tive to cause the Bureau of Drugs to want to experiment in its own 
policies regarding post-marketing surveillance. The motivating 
force of funds (not otherwise available to FDA through its own 
budget) to do policy experiments unquestionably gave us the opportuni- 
ty and the where-with-all (sic) to do things we would have liked to 
do but were unable to do otherwise. In addition to the funds, the 
brain power, interest, enthusiasm, and dedication of the ETIP staff 
members who participated directly in the project over the last six 
years and indirectly through its on-site evaluation contractor, PDI, 
have enriched our project advisory group’s capability to design 
better projects, experiments, and studies; to evaluate and under- 
stand in a more meaningful way the results of these studies and 
projects; and to use this knowledge to formulate more rational, 
effective, and efficient siden regarding post-marketing surveil- 
lance and drug safety.... 


Federal Communications Commisstont 


In the 1979 report by ETIP’s evaluation panel of outside experts, David A. 
Irwin of the FCC echoed Halperin’s comments about "the mating dance of the 
porcupines" in describing the development of an ETIP-FCC partnership of mutual 
agreement, understanding, trust, and respect, and the value of that partner- 
ship: 

As a result of our work with ETIP, we have done some things 


we would not have done and have perspectives we would not 
have had. 


1. This and the remaining sections of this chapter are derived in large 
part from Op. Cit., p. V-l, pp. 5-11. 


ETIP, he told the panel, helped the FCC look at its processes during a turbulent 
period of technological growth in the communications industry and provided the 
means for assessment, evaluation, and prediction of what would happen next that 


would require or be ameliorated by FCC action. 


Federal Supply Service 

When a Department of Commerce evaluation team canvassed agencies about 
their involvements with ETrP, | FSS officials credited ETIP with providing 
support needed to maintain innovative purchasing procedures in the face of 
persistent opposition from tradition-minded staff. ETIP contributions cited 


as particularly valued included: 


o Funds. FSS found it necessary to seek funds outside its own budget 
to support experimental purchasing. 


o Technical Assistance. Although ETIP staff shortage limited the 
flow of technical assistance, FSS management officials found ETIP 
technical advice extremely valuable in starting the early LCC and 
VIC experiments. Operational staff, however, disputed the 
statement. : 


o Leverage and prestige. ETIP’s position as an outside independent 
agency helped FSS maintain its commitment to innovative procure- 
ment, inasmuch as decisions relative to that commitment could not 
be made entirely internal to the agency- In addition, ETIP’s 
affiliation with the highly regarded NBS served to some extent to 
blunt suspicion of the experiments both outside and within FSS. 


Securities and Exchange Commission 


The SEC also praised ETIP’s facilitator role in the effort to develop a 
data-gathering, monitoring, and analysis model within the agency. Citing results 


from its applications of the model, SEC reported: 


l. Op. Cite. p- IIiI-6, Kelly. 
Zs Op. Gites TLI-40, pe 8. 
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The research findings, with the corresponding regulatory implications, 
obviously could not have been produced without the customized library 
of computer programs.-.-(one of the components) of the monitoring 
capability developed as part of the ETIP joint project. 


Office of Technology Assessment 


John Young of OTA, who was involved with ETIP from its beginning, told 
the 1979 evaluation panel that ETIP’s greatest assets were that it served as 
a non-threatening outside stimulus to line agencies, that it reduced the risks 
involved in doing something different, and that it provided both expertise and 
seed money for the development and assessment of innovations. He also noted 
that ETIP’s placement in the NBS enhanced its nonthreatening character. 

Young said OTA benefited from and relied on ETIP’s work -- for example, on 
its design for cooperative agreements -- and described the relationship between 
the agencies as symbiotic. He cited the two agencies” mutual recognition of 
the importance of policy and administrative experiments and of addressing 
process and management issues. The significance of ETIP’s work, he noted, 
rested partly on the fact that policy dececting innovation is being made all 


the time and needs to be monitored. 


Ze The ETIP Strategy 


ETIP’s approach was almost unique in the history of government, relying as 
it did on the building of collaborative relationships between agencies for the 
purpose of experimentation with policies and procedures. On the basis of its 
sampling of agency reactions to ETIP, the 1979 panel identified several princi- 
ples which derived from this strategy and appeared to underly ETIP’s effective- 


ness across a variety of activities. (No order of priority is implied in the 


following observations.) 


Partnership 

In many ways, ETIP behaved like a consulting firm with other agencies 
as its clients, but a more complete joining of ETIP’s and the client-agency’s 
resources resulted than would have been the case in the typical consultant- 
client relationship. For this reason, the relationship could best be described 
as partnership, a concept that extended to relations with project contractors, 


which were jointly selected and managed by the agency and ETIP. 


Choice of Partners 

Of the many kinds of experiments which could have been undertaken, ETIP 
selected those for which it could find a willing partner, the "target of 
opportunty" strategy. In applying this strategy ETIP joined with agencies 
which were willing and ready to work with it. This contributed much towards 
ensuring the success of ETIP efforts within an agency, and successes generated 
in this way enhanced and spread ETIP’s reputation as a valuable resource, 
thereby contributing to the readiness and willingness of other agencies to 


accept ETIP as a partner. 


Staff-to-Staff Collaboration 

ETIP dealt with the concerned staff of other agencies and made no effort 
to acquire "authority" or "clout" by starting at the top or by other methods. 
Instead, ETIP earned its credibility in the process of building a relationship 
with a new partner, sometimes with the assistance of testimonials from other 


partners. 


Seed Money 


Even though the staff of a line agency may think that some changes might 


be desirable, it is often difficult to draw funds away from present uses into 
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uses likely to be perceived as risky or of uncertain value. ETIP’s ability to 
fund explorations and experiments designed to determine more clearly the need 
for and describe more clearly the nature of contemplated changes was vital to 
the initiation of work with a new partner. 
Out of initial collaboration evolved commitment of the agency—partner to 
the contemplated changes and to provision of its own money and staff for continu- 


ing the effort. 


Non-Emergency Role 


The process of building relationships, establishing the need for changes, 
identifying the potential changes needed, designing and implementing them, ‘and 
evaluating the outcomes, was not a short-term undertaking. To see this process 
through, ETIP needed protection from being drawn into the role of fire-fighter 
or trouble-shooter.e Its independent status within the NBS was of great help 
in maintaining such a posture (although, as has already been noted, efforts 


frequently were stymied by unrealistic time limitations nevertheless). 


Transferral of Skills and Ownership 

ETIP brought to its partners considerable skills in experiment design, 
technical analysis, and evaluation, as well as new perspective and the 
collaborative approach. Through its work with ETIP, each partner became 
familiar with these skills and became increasingly able to apply them itself. 
Thus ETIP was able to gain the advantages of a small team that could increase 


its impact by reproducing itself in its partner agencies. 


Adaptability 
Although ETIP devoted considerable effort to documenting its learning and 


sought useful generalizations, it refrained from using one model to approach 


all problems. It was problem-centered and client-centered and was thereby 


able to adapt its approach to the uniqueness of each situation. 


Modesty 


ETIP refrained from taking over or telling a partner what to do. Instead, 
much as in a therapist-patient relationship, it worked with the partner in a 
problem-solving mode until both discovered what actions were needed. Similarly, 
ETIP did not use its knowledge and skills to keep its partner dependent but 
freely shared both knowledge and skills. Finally, ETIP did not seek credit for 


its partners” successes. 


Development of Commonality 


An important part of ETIP’s expertise was knowing how to develop common 
goals, common language, common understandings, and common strategies with 
each of its partners and stakeholders. An example of the application of this 
knowledge was incorporation of the Joint Commission on Prescription Drug Use 
(JCPDU) and its agenda into the ETIP/FDA ear er ouitn and in the protracted 
pre-experiment negotiations in which the need for thoroughly briefing FDA 


personnel on ETIP’s process and evaluation interests was addressed. 


Synergic Stratepies 


ETIP avoided the use of adversarial strategies. It was neither advocate, 
nor activist, nor salesman. It recognized that the only source of creativity 
in the relationship lay in the differences between itself and its partners, and 
put those differences to work in developing new insights, perspectives, and 


strategies. 
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C. CONCLUSION 


At the outset it was stated that ETIP could be assessed from each of three 
vantage points -- by how well it could answer each of three questions. 


Following, then, is a summation of ETIP’s answers to those questions: 


Lis Did ETIP stimulate private technological initiatives? 


New products that can be attributed directly to ETIP interventions in 
government-industry processes were admittedly modest in number and significance, 
but ETIP did demonstrate that monitoring the private-sector impacts of government 
activities and modifying those activities based on analyses of the results of 


that monitoring can and will foster private-sector technological advance. 


Zs Did ETIP alter government agency policies and procedures? 


Institutionalization of monitoring for private sector impacts, of evaluat- 
ing those impacts, and of basing incremental policy and procedure changes on the 
results of those evaluations was institutionalized at a Gider vari eee of agencies, 
formally altering those agencies’ perceptions about their roles and how to 
manage them. Whether these perceptions would outlast ETIP and continue to spawn 


new modifications as needed was to be hoped for. 


oe How much did ETIP learn about change in government and industry? 


ETIP derived from its experiments a breadth and depth of new knowledge 
about the complexities woven into the fabric of government-industry interaction. 
This knowledge filled to a considerable extent the vacuum perceived in the early 


1970s and provided a basis upon which future government decision-making can and 


should be made. 


Above all, ETIP learned through the trial and error processes of its 
collaborative strategy of administrative experimentation that its strategy was 
a successful way to achieve change in government line-agency policies and pro- 
cedures. 

An ETIP-like process is vital to government if that government hopes to 
keep abreast of an ever more rapidly changing world. Policy that affects 
innovation in the private sector is being made continuously by government and 
requires monitoring and modification, but agencies of government have not yet 
learned this basic lesson that ETIP learned, the lesson that undergirded all 
of the program’s activities. 

At ETIP’s end a large task -- part learning and part application -- 
remained to be accomplished. It is likely that it can only be done under the 
aegis of an organization like ETIP, a third-party agency capable of sparking 
formation of collaborative problem-solving partnerships. 

In 1982 there was no reason to expect that government would soon institute 
another such effort or agency, or that Wed: agencies would learn the necessary 
lessons, develop the necessary perspectives and approaches, or form the necessary 
collaborative partnerships unless such an agency were created as a continuing 
resource to facilitate those developments. 

ETIP ended just when its efforts were beginning to pay off. Once dis- 
mantled, it is likely to take many years of renewed effort before ETIP’s kind 
of expertise will again be internally available to government. 

Such an agency requires a home where protection of funds, time-perspective, 
autonomy, and freedom from political and bureaucratic interference can be pro- 


vided. NBS provided such an environment for several years. 
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Should government again discover the need to experiment with change and 
decide to reestablish an ETIP-like agency, various alternative potential homes 
for its activities should be explored, including establishment of an independent 
existence as a quasi-government entity. Such questions, however, address 
themselves to future policy-makers once a commitment to experimentation with 
government policies and processes is reawakened by the rediscovery of need. 


Necessity remains the mother of invention. 
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How to Locate ETIP Records 


In the following pages of this appendix, the individual projects of the 
Experimental Technology Incentives Program (ETIP) are listed. The projects are 
listed in numerical sequence. All projects when they were initiated were assign- 
ed a number. Some of these projects were never implemented, and this fact is 
noted where appropriate. After this list is found a compilation of the reports 
associated with these numbered projects. Readers of this final report may wish 
further detail on the projects described in the preceding pages by ordering any 
of these reports. If further detail is needed (or if the person has difficulty 
in ordering any of the reports listed), the National Bureau of Standards (NBS) 
should be contacted. General inquiries can be handled by the Inquiry Service of 
the Technical Information and Publications Division (Area Code 301/921-2318) or 
the Public Information Division (921-3112). For technical detail and data, 
interested persons should contact Mr. Richard Penn of the National Engineering 
Laboratory at the Bureau (921-3324). Mr. Penn is the former Acting Director of 
the Program and is intimately familiar with most of the individual projects. 


Technical data for each of the projects are found at increasing levels of detail 
in the records system established at the termination of ETIP. These levels are: 


1. The final ETIP report and other project reports. 
2. Project overviews for each of the projects. 
3. Individual project files. 


Interested persons would be advised to begin with published reports. If further 
detail is needed, the next step is the project overview. Mr. Penn has a copy of 
each of these overviews. In addition, each individual project file contains a 
copy of the overview for that project. Each overview includes a brief descrip- 
tion of the project, its objectives, the approach taken, financial history, 
lessons learned, and resulting reports. If even greater detail is desired, 
interested persons can access the individual project files by contacting any of 
the offices listed above. Until the spring of 1983, these files will be located 
physically at NBS. After this, they will be removed to the Federal Records 
Center in Suitland, Maryland. Despite this move, all inquiries should continue 
to be made to NBS, which alone has the authority to acquire the individual 
project files. 


In most cases, the reports listed below are obtainable through the National 
Technical Information Service (NTIS). In some projects, a decision was made not 
to publish certain reports. In any case, all reports are filed in the individual 
project files. 


List of ETIP Projects by Number 


Projects have been divided into the categories of Regulation, Procurement , 


Research and Development (R&D), Small Business, Economic, Assistance, 


Experimental Methods, Office of Strategy and Evaluation (OSE), and Other. 
Projects were placed into these categories either by (1) the ETIP area which 
generated them or (2) in cases where no area was identified, by the content of 


the project. 
category or was unknown. 


"Other" has been used for projects where content did not fit any 
The OSE category refers to projects generated by the 


OSE group which was added to ETIP to support Department of Commerce initiatives 


in technological innovation. 
categories. 


Number 


CON DU F WH 


Zs 


24 


25 


26 


Title 


Power Lawn Mower Procurement Experiment 

Room Air Conditioner Procurement Experiment 
Frostless Refrigerator Procurement Experiment 

Home Water Heater Procurement Experiment 

Kitchen Range Procurement Experiment 

Agency Initiated Design of Procurement Experiments* 


Research Effort in Fiber, Textile, and Apparel Flammability 


Analysis of the Dynamics Underlying Regulatory Changes 
Having a Significant Effect on Innovation 
Imperfections in Capital Markets 


Analysis of Reasons for Nonacquisition of Independent 
Inventions by Corporations and Patent Broker/Developers* 
Analysis of Federal Demonstration Projects 

Commercial Potential of Federal Invention Disclosures 
Rejected for Patent Filing* 

Application of University Research Output 

Guidelines for Industrial R&D Aggregation* 

Enhancing the General Welfare of U.S. Industry* 
Development of Models for Selecting Technology-Stimulating 
Policy Tools* 

Federal Contingency Planning for Anticipated Technological 
Crisis 

Foreign Research Development Corporations* 

Federal Funding of Civilian R&D 

Creation of National Product Marketing Service* 

A Survey of Management's Opinion About the Relationship 
Between Regulation and Innovation* 

Use of Regulatory Authority to Stimulate Nonmandated 
Technological Change* 

An Examination of Methods Which May Be Used to Stimulate 
the Adoption of Regulations Which Provide an Incentive 
for Innovation* 

Assistance in the Design of a Federal Institution for 
Stimulating Private Sector R&D* 

Analysis of Barriers to Technological Change for ETIP 
Procurement Experiments* 


Development of Models for Achieving Mandated Technological 
Change* 


.. OSE projects have not been subdivided into narrower 
An asterick (*) means that the project was never implemented. 


Procurement 
Procurement 
Procurement 
Procurement 
Procurement 
Procurement 
R&D 
Regulation 


Economic 
Assistance 
R&D 


R&D 
R&D 


R&D 
R&D 
R&D 
Other 


Economic 
Assistance 
R&D 

R&D 

Other 
Regulation 


Regulation 


Regulation 


R&D 
Procurement 


Regulation 


Number 


af 
28 
29 
30 
eH 
32 
ae 
34 
35 
36 
37 
38 


39 


Title 


Development of Procedures for Evaluating Technology-Based 
Firms for SBA Funding* 

Assisting Small Firms Required by Regulation to Undergo 
Technological Change 

Conference on Federal/Industrial Partnership in New Product 
Deve lopment 

Intergovernmental Conference on Public Civilian Technology 
Policy* 

Deregulation of Rail Produce Shipments 

Connecticut Product Development Corporation 

The Relationship Between R&D Funding and Environmental 
Regulation* 


Providing Incentives for Large Users of Electrical Energy to 


Use It More Efficiently* 

Life-Cycle Costing Applied to the Procurement Process* 
LCC Guide for Evaluating Building Design Alternatives 
Increasing Health Care Productivity Through an Improved 
Selection of Technologies* 

Design of Federal Procurement Experiments for Health Care 
Delivery 

Development of Guidelines for the Use of Performance vs. 
Design Specifications in Procurement* 

Design of Three Procurement Experiments 

Design of Additional Federal Procurement Experiments 
Systematizing the Use of Federal Procurement Incentives in 
Federal Purchasing 

Government Market Research for Procurement Guidance* 
Experiment Design for Industry-Specific Regulation 
Experiment Design for Process-Oriented Regulation 
Competency Evaluation of Small R&D Firms 


Accelerating the Licensing of Nuclear Power Reactors 

An Experiment in the Regulation of Pesticides 

Regulating the Irradiation of Foods* 

Regulation of Renewable Resources* 

Design of Regulations for Public Utility Commissions* 
Regulating New Pharmaceuticals* 

Government/Industry Joint Venture in R&D* 

Accelerating the Use of Solar Energy Research* 

Technology Impact Statements* 

Geothermal Energy Technology Delivery System* 
Intersectoral Transfer of a Building Technology by a Major 
User* 

Automobile Tires* 

Acquisition of High Speed Printer Ribbon on a Life-Cycle 
Basis 

Public Market Aggregation as an Incentive for Technological 
Change 

Biomedical Monitoring Systems (BMS) Procurement Experiment* 
Blood Tubes Procurement Experiment* 

Internally Oriented Life-Cycle costing (LCC) Program for 
Federal Purchasing 


ETIP Area 


Small 
Business 
Small 
Business 
Procurement 


R&D 


Regulation 
R&D 
Regulation 


Other 


Procurement 
Procurement 
Procurement 


Procurement 
Procurement 


Procurement 
Procurement 
Procurement 


Procurement 
Regulation 
Regulation 
Small 
Business 
Regulation 
Regulation 
Regulation 
Regulation 
Regulation 
Regulation 
R&D 

Other 
Regulation 
Other 
Procurement 


Procurement 
Procurement 


Procurement 
Procurement 


Procurement 
Procurement 


Number 


A-4 


Title 


Value Incentive Contracting Program for Federal Purchasing 
Utilizing University Research in Transportation* 

Technical Evaluation* 

New Modular Integrated Utilities System (MIUS) Application 
Project 

Institutional Evaluation Through Hindsight-Tracer Studies 
on Educational R&D Products and Programs* 

An Examination of the Cooperative Agreement as an Element 
in Technology Delivery Systems* 

Payment of Costs Incurred by the Federal Supply Service in 
Providing Background Information and Conducting Preliminary 
Studies on Potential ETIP Projects 

Acquisition of General Purpose Hard Surface Cleaning Agents 
on a Performance Basis 

Technical Assistance to the Procurement Policy Area of ETIP 
Oscilloscopes* 

Procurement of Upholstered Furniture Using Performance 
Specifications 

Systematizing Local Procurement as the Preface to 
Technological Change 

Experiments in Computer Applications to Electric Rate 
Making-Lag Reduction 

Symposium in Assistance to Small Technology-Based Firms 


Data Collection, Summarization, and Interpretation for ETIP 
Project with Connecticut Product Development Corporation 
Survey of Five-Year Results of Innovation Loan Program 


Systematizing the Design, Development and Use of Incentives 
in the Procurement of Medical Equipment and Supplies 
Design of Federal Procurement Experiments for Improved 
Health Care Delivery 

Office Copiers* 

Information Management in Procurement* 

Regulation, Competition and Innovation* 

Using Technology to Improve the Data and Timeliness of the 
Regulatory Process for Occupational Health and Safety 
Experiment in Contractor Selection* 

Management of Federal Civilian Research and Development 
Programs 

Experiment in Drug Lag Reduction 

The Connecticut Experiment* 


Evaluation of Federal Supply Service Procurement Experiments* 


Evaluation of ETIP Procurement Experiments at the State and 
Local Levels under ETIP Projects No. 60 and No. 75* 
(Not Used)* 


Federal Financial Assistance and Technological Change 


Evaluation of ETIP 

Evaluation of Project "Accelerating the Establishment of 
Standards by the Nuclear Regulatory Commission (ETIP #47) 
Real-Time Case History of ETIP Project 67: Integrated 
Utility System 


Procurement 
R&D 

Other 

R&D 


R&D 
R&D 


Procurement 


Procurement 


Procurement 
Procurement 
Procurement 


Procurement 
Regulation 


Small 
Business 
R&D 


Small 
Business 
Procurement 


Procurement 


Procurement 
Procurement 
Other 
Regulation 


Other 
R&D 


Regulation 
R&D 
Procurement 
Procurement 


Economic 
Assistance 
Other 


Experimental 


Methods 
R&D 


Number 


97 


98 

21 
100 
101 
102 
103 
104 
105 
106 
107 
108 
109 
110 
Lid 
1i2 
ir 
114 
115 
116 
ie 


Title 


Evaluation of the Inter-Organizational Consortium for 
Research and Development on Flammable Fabrics 

Improving the Standard Setting Process* 

Expanded Nuclear Standards Project* 

Regulation of Competition* 

Anti-Trust Boundary Project* 

Regulatory Degree Project* 

Evaluate Standards Project* 

Evaluating Economic Regulation* 

Evaluating Certification Projects* 

Confirming Procurements (FSS)* 

Design of Procurement Experiment s* 

Monitoring Assistance* 

Federal Procurement of Health Related Goods (Eval)* 
Space Procurement (Eval)* 

Market Planning Project* 

R&D Projects (Eval)* 

Test Loan Evaluation Procedure* 

Test Small Business Regulatory Compliance Procedures* 
Exploratory Analysis* 

Case Studies in Procurement 

The Implementation of Life-Cycle Costing at Federal Supply 
Service--An Evaluation Plan for the Agency Effects of the 
Implement at ion* 

Preparation of a Case Study on the Procurement of Window 
Air Conditioners 

Evaluate LCC Seminars 


Commercial and Technological Impacts of the Procurement 
Experiments 

The Agency and Limited Commercial Effects of the ETIP 
Procurement Experiments 

The Agency Effects of Policy Changes Implemented on the 
Basis of ETIP Procurement Experiments* 

Case Studies in Procurement-Value Incentive Clause 
Technical Assistance to Procurement Program Area of ETIP 
Symposium for the Dissemination of Results from the Team 
Project for the Development of Flame Resistant 
Polyester/Cotton Fabrics 

An Expanded Program for the Dissemination of Results from 
a Team Project for the Development of Flame Resistant 
Polyester/Cotton Fabrics 

Workshops on Technology and the Law 

Campus-Community Integrated Utility System* 

Procurement of Paint* 

Survey on the Role of State Legislators in Procurement 
Policy 

Warrant y* 

Refinement and Dissemination of Policy Issues for 
Anticipated Crises in Commodity Supplies 

National Conference of State Legislatures* 

Evaluation of Consortium Results 


ETIP Area 
R&D 


Regulation 

Regulation 

Regulation 

Regulation 

Regulation 

Regulation 

Regulation 

Regulation 

Procurement 

Procurement 

Other 

Other 

Other 

Other 

Other 

Other 

Other 

Other 

Procurement 
Experimental 

Methods 


Procurement 


Experimental 
Methods 

Experimental 
Methods 

Experimental 
Methods 
Other 


Procurement 
Procurement 
R&D 


R&D 


Regulation 
R&D 
Procurement 
Procurement 


Procurement 
Economic 
Assistance 
Procurement 
R&D 


Number 


135 


136 


A-6 


Title 
Expansion of the Energy Report 
Small Business (Design of Analysis) 


Directed Communication Services for the Procurement Program 
Area of ETIP 

Forest Service Tool Sheath 

Use of Procurement Incentives in the Purchase of Headlamps 
Development of Experiment in Communications Regulation 
Development of Surgical Glove Specification and Preparation 
of Case Study 

Technical Assistance in Procurement Experiments 

Hypotheses Concerning the Influence of Department of 
Defense Sponsorship of Electronics Research and Development 
on the Development of the Civilian Electronics Industry 
Dissemination of Analysis of Federal Demonstration Projects 
Study 

Solar Economic Incentives: Policy Research Through 
Retrospective and Experimental Analysis 

Commercial Commodity Acquisition Program 

Warranties (FSS) 

SEC--A Monitoring System for Effective Regulation of 
Venture Capital Markets 


Evaluation Potential for Regulatory Lag Reduction Experiments 


Prototype Purchase - Lawn Mowers 

Developing Experiments in Occupational Health and Safety 
Regulation 

Continued Development of Systems for the Evaluation of the 
Agency and Commercial Impacts of ETIP Experiments* 
Experiments in Noise Regulation 

Evaluation of Communications Regulation Experiments 
Feasibility 

R&D Expenditures, Productivity, and Economic Growth 
Provide Technical Assistance for the PHS Experiment 

Data Collection and Analysis Support for ETIP Regulatory 
Experiments 


Pesticide Regulation Experiment 

Development of an ETIP Communications Program (Washington 
Marketing Group) 

Improving Rule Making at Federal Trade Commission 
Communications Deregulation Experiment 

Assistance to ETIP in Establishing Effective Interaction 
with Key Officials of Office of Federal Procurement 
Policy (OFPP) 

Implementation of Project 76 RAm Model in Boston Edison 
Rate Case 

R&D Technological Change and Economic Growth 

Energy Intensive Products - LCC 

Federal Policies to Develop the Interface Between Public 
Utilities and Dispersed, Small-Scale Solar Energy 
Systems--DOE 


ETIP Area 


Economic 
Assistance 
Economic 
Assistance 
Procurement 


Procurement 
Procurement 
Regulation 

Procurement 


Procurement 
R&D 


R&D 


Economic 
Assistance 
Procurement 
Procurement 
Economic 
Assistance 
Regulation 
Procurement 
Regulation 


Experimental 
Methods 
Regulation 
Regulation 


R&D 
Procurement 
Experimental 
Methods and 
Regulation 
Regulation 
Procurement 


Regulation 
Regulation 
Procurement 


Regulation 


R&D 
Procurement 
Economic 
Assistance 


Number 


167 


168 


169 


170 


171 


a 


ie 


174 


BID 


176 
Lis 


178 
179 
180 
181 
182 
183 


184 
185 
186 
187 
188 
189 
190 


191 


es 


295 
194 


195 
196 
197 


Title 


Federal Policies and Private Investment in Technological 
Change 

Taxonomy of Incentives 

Analytical Support and Data Collection in Regulatory 
Administrative Experiments 


Analytical Support and Data Collection in Procurement 
Administrative Experiments 

Preparation of Issues Paper on Product Remanufacturing 
Investigate the Cumulative Impact of the Government 
Regulations on the Innovation in the U.S. Steel Industry 
from 1945 to the Present Time 

Summary and Synthesis of Regulatory Experimentation 

Case Studies 

Development of Issues Papers for a Policy Review of 
Industrial Innovation 

Preparation of Reports for the Public Interest Group 
Advisory Committee 

Technological Innovation and the U.S. Economy 

The Role of the State and Local Government in Technological 
Innovation: A Conceptual Paper 

Industrial Innovation - The State and Local Government Role 
Labor's Role in Industrial Innovation 

Technology and the U.S. Economy: International Dimensions 
Conduct of Industry Survey on Multiple Award Contracts 

Air Cargo Deregulation Experiment 

Operation Breakthrough, OMB Circular A-109 and Acquisition 
Differences 

Conference of Small R&D and Technically Based Companies on 
Opportunities for Small R&D Companies in the Federal 
Government ' 

Project Plan for Quiet Products 

Research on the Pattern of U.S. Growth Relative to Its Major 
Competitors 

Colloquium: Industrial Innovation and Public Policy Options 
Evaluation Process for Environmental Regulation Development 
Market Place Incentives for Air Pollution Reduction 
Potential Problems and Barriers to the Development of a 
VLSI-Based Technology for the Semiconductor Industry 

(A) GSA Implementation of Domestic Policy Review 
Recommendat ions* 

(B) Alternative Approaches to National Technological Goals 
Study: Adversarial and Cooperative Relationships Between 
Business and Government in Four Countries 

Analysis of State-Operated Sources of Venture Capital 
Assessment of Factors Affecting the U.S. Semiconductor 
Industry and Identification of Possible Government 
Intervention to Ensure the Viability of the Industry* 
Patents and the Technological Innovation Process* 

Railroad Management Innovations for New Regulatory Era 
Research Support of the Center for the Utilization of 
Federal Technology 


ETIP Area 


Economic 
Assistance 
Regulation 
Experimental 
Methods and 
Regulation 
Experimental 
Methods 

OSE 

OSE 


Regulation 
OSE 
OSE 


OSE 
OSE 


OSE 

OSE 

OSE 
Procurement 
Regulation 
Procurement 


OSE 


Procurement 
OSE 


OSE 
Regulation 
Regulation 
OSE 


Procurement 


OSE 
OSE 


OSE 
OSE 


OSE 
Regulation 
OSE 


Number 


198 
eee) 
200 


208 
202 
203 
204 
205 


206 


Title 


Study the Economics of the Semiconductor Industry 
Strategic Analysis of the U.S. Semiconductor Industry* 
Background Study and Criteria to Develop and Establish 
Fellowship Program in CUFT, DOC 

Study: Patent System for the 1990's 

Study: Decade of Patent Innovation 

Study: Industrial Property Mechanisms 

Study: Restructuring of the Present System of Patents 
and Copyrights 

Enhancing the Competitiveness of the U.S. Semiconductor 
Industry* 

Evaluation of Deregulation Trends in the Rail Industry 


ETIP Area 


OSE 
OSE 
OSE 


OSE 
OSE 
OSE 
OSE 
OSE 


Regulation 


A List of Published ETIP Reports 


The following is a list of ETIP reports which have been published by the National Bureau of Standards (NBS), the National 
Technical Information Service (NTIS), or the Government Printing Office (GPO). NBS numbers are NBS-GCR-ETIP or as 
indicated. NTIS numbers are PB or COM, or other sources as indicated. 


Project 
Number Title Author Date NBS No. NTIS No. 
1 Experimental Technology Incentives Program Booz~Allen October 73-04 COM-74-10256/7Gl 
Procurement Experiment Plan 1: Stimulating Noise | Applied Research 1973 
Control Technology in Consumer Products Through 
Federal Government Procurement Action 
2 Experimental Technology Incentives Program Booz-Allen November 73305 COM-74-10257/5G1 


Procurement Experiment Plan 2: Stimulating Applied Research 1973 
Efficiency and Noise Abatement in Room Air 
Conditioners Through Federal Government 
Procurement Action 
3 Experimental Technology Incentives Program 
Procurement Experiment Plan 3: Stimulating 
Increase Efficiency in Frostless Household 


Refrigerators Through Federal Government 


November COM-74-10258/3G1 


1973 


Booz~Allen 
Applied Research 


Procurement Action 


A Survey of Manufacturers’ Views on the ETIP P. C. Goodman December PB253242 

Procurement Experiment, Volume One: Refrigerator-] NBS 1975 

Freezers i 
4 A Survey of Manufacturers' Views on ETIP P. C. Goodman January PB251213 

Procurement Experiment, Volume 2: Water Heaters NBS 1976 

ETIP Procurement Experiment Plan 4: Stimulating Booz-Allen and November 

Increased Efficiency in Residential Water Heaters | Hamilton, Inc. 173 

Through Federal Procurement Action ae 
5 A Survey of Manufacturers’ Views on the ETIP P. C. Goodman February PB253243 

Procurement Experiment, Volume 3: Ranges NBS 1976 

ETIP Procurement Experiment Plan 5: Stimulating -Booz-Allen and November 

Technological Improvements in Household Ranges Hamilton, Inc. 1973 


Through Federal Government Procurement Action 


Project 
Number 
7 


i} 


17* 


Title Author Date NBS No. 
Report of Activities Carried Out Under Contract R. H. Barker 75-01 
4-35963 "Fiber, Textile and Apparel Research - J et eeoLor. oe 
Cotton Polyester Blends", July 1974-75 Clemson 
University 
Development of Flame Retardants for Polyester/ R. H. Barker September 16222 
Cotton Blends M. J. Drews 1976 


Clemson 

Universit 
Charles River 
Associates, Inc. 


February 76-08 
1976 


An Analysis of Venture Capital Market 
Imperfections - Executive Summary 


An Analysis of Venture Capital Market 76=12 
Imperfections - Final Report 

Analysis of Federally Funded Demonstration April 76-05 
Projects - Executive Summary 1976 

Analysis of Federally Funded Demonstration 76-06 
Projects - Final Report ; 

Analysis of Federally Funded Demonstration 76-07 


Projects - Volume 3, Supporting Case Studies 
A Framework for Analyzing Commodity Supply 
Restrictions 


Charles River 
Associates, Inc. 


August 
1976 


August 
1976 


Cartelization in the World Cobalt Market: 
Economic Analysis and Policy Implications 


August 
1976 


Policy Implications of Producer Country Supply 
Restrictions: The World Platinum and Palladium 
Markets 


September 
1976 


Policy Implications of Producer Country Supply 
Restrictions: The World Manganese Market 


NTIS No. 


COM-75-11369 


PB257884 


PB254078 
PB254996 
PB253108 
PB253918 


PB262123 


PB258093 
PB264386 


PB264387 


PB264388 


OI-V 


Project 
Number 
17* 


‘Me 


28 


a 


Title Author Date NBS No. 
Policy Implications of Producer Country Supply August 
Restrictions: The World Copper Market 1976 


Policy Implications of Producer Country Supply September 
Restrictions: The World Chromite Market 1976 


Policy Implications of Producer Country Supply Charles River September 
Restrictions: The World Aluminum/Bauxite Market Associates, Inc. 1976 


Policy Implications of Producer Country Supply November 
Restrictions: The World Energy Market 1976 


Policy Implications of Producer Country Supply February 
Restrictions: Overview and Summary 1977 


* Available as a set under NTIS No. PB264385 
Federal Funding of Civilian Research and Arthur D. Little | February 
Development, Volume I, Summary 1976 


Federal Funding of Civilian Research and 76-04 
Development, Volume II, Case Stud 


Impact on Small Business of Government Regulations| Charles River September 
Which Force Technological Change Associates, Inc. Lo75 


A Long Term Study of Produce Transportation Manalytics, Inc. December 
ike Wav 
Volume 1, Executive Summary 
Volume 2, General Summary 
Volume 3, A Profile of Fresh Fruit and Vegetable 
Shipments from Selected Growing Areas 
Volume 4, A Profile of Fresh Fruit and Vegatable 
Marketing and Distribution in Selected 
Destination Cities 
Volume 5, Fresh Fruit and Vegetables - 
Transportation Services and Costs : 
Volume 6, Transportation Equipment Technology 
Volume 7, Fresh Fruit and Vegetable Packinghouse 
Operations and Economics_ 


NTIS No. 
PB264389 


PB264390 


PB264391 


PB264392 


PB264393 


PB251266 


PB251683 


GPO SN 045-000- 
00144-0 

Federal Rail 
Administration 


FRA OOPD 78-2.1 
FRA OOPD 78-2.2 
FRA OOPD 78-2.3 


oO 


FRA OOPD 78-2. 


FRA O0PD 762. 


Nn 


FRA OOPD 78-2.6 
FRA QOPD- 78-27 


Project 
Number 
3] 


45 


47 


48 Incentives for R&D in Pest Control (Vols. I & II) eau Den Little eee ee 
1976 


Title 


Volume 8, Evaluation of Logistics Alternatives 
Volume 9, Fresh Fruit and Vegetable Commodity/ 


Origin/Destination Selections 


A Long 


Volume 
Volume 11, 


Description 


Term Study of Produce Transportation 


10, Produce Shipment Description Model 


Glossary of Produce Marketing and 


Distribution 


Interim Report on Potential ICC/ETIP 


Administrative Experiments (under Project 169) 


Vegetables, 


Exempt Rail Transportation of Fresh Fruits and 


Initial Impacts 


Economic Aspects of Joint R&D Ventures and the 
Private Sector (see Project No. 
of other work done in Project No. 


173 for Summary 
45) 


An Experiment on Accelerating the Writing of 
Nuclear Standards Conducted for the American 
Standards Institute 


An Assessment of an Experiment Accelerating the 
Development of Nuclear Standards (done under 


Project No. 


157) 


Author 


Manalytics, Inc. 


M. Mulkey 
W. Frederick 
Performance 
Development 
Institute 


D. Fulmer, NBS 


Manalytics, Inc. 


S. Buchanan 
Public Interest 
Economics Center 
N. C. Wittenbrock 
Battelle 


J. P. Woodward 
Performance 
Development 
Aa es 
Garrity, NBS 


Date 


December 
1977 


1978 


September 
1975 


NBS No. 


80-85 


79569 


NTIS No. 
FRA OOPD 78-2.8 
FRA OOPD 78-2.9 


Federal Rail 
Administration 


FRA OOPD 78-2.19 
FRA OOPD  /8—27 11 


PB80202375 


ICC, Office of 


Policy and Anaysis, 


Contract No. 80-M- 
1159 
PB80208861 
PB256129 
PB80202104 


EPA C540/9-77-008 


iZ1-v 


Project 
Number 


48 


63 


67 


70 


76 


Title Author Date NBS No. 
Developing Incentives for Pest Control Methods, Arthur D. Little {| January 76-34 
Proceedings of a Conference, October 26-27, 1976 1977 


(also see Project No. 158) 
Life Cycle Costing Procurement, Case 1, Room Air 
Conditioners 


July 
ES75 


Logistics 
Management 
Institute 


Logistics 
Management 
Institute 


July 
i373 


Life Cycle Costing Procurement, Case 2, Water 
Heaters 


January 
1976 


Life Cycle Costing Procurement, Case 3, Gas Ranges 


May 
1977 
December 
1976 


Life Cycle Costing - Case Study (A Compilation 
of Previously Issued Studies) 
A Guide to the Dollar Savings Potential in 
Integrated Energy/Utility Systems 


Reynolds, Smith 
and Hills 


October 
1976 


Feasibility Study and Conceptual Design at the 
University of Florida 


December 
1976 


Feasibility Study and Conceptual Design at 


February 
T977 


A Guide to the Available Technology for 
Integrated Energy/Utility Systems 


P. L. Geiringer October 


A Guide to Community Planning for Integrated 


Energy Utility Systems and Associates 1978 

Proceedings of Procurement Practices Symposium, Tee e30 iFody May NBSIR 

Federal, State, and Local, January 28-30, 1975 J. G. Berkey NBS POP TA0 
Overview Report: Experiment in Computer Temper, Barker & September 79-70 

Applications for Regulatory Agencies Sloane, Inc. 1979 

Regulatory Lag: Administrative Causes and September CES ae: 


1979 


Solutions 


NTIS No. 
PB263275 


PB253475 


PB253476 


PB253477 


PB268162 


PB266042/1WE 


PB266043/4WE 


PB266044/ 7WE 


PB269042/1WE 


PB290675/AS 


COM7511210 


PB300770 


PB80146776 


Project 
Number 
76 


7a 


vo 


Titie 
Regulatory Analysis Financial Model RAM 
Descriptive Documentation, Vol. 1 


Performance Evaluation Model: Suggestions for 
Use and Description of the Model 


Use of A Future Test Year for Electric Utility 
Ratemaking 


Use of A Future Test Year for Electric Utility 
Ratemaking, Appendices A-D 


Automatic and Discretionary Adjustment Procedures 
for Electric Utility Rates, Volume I 


Automatic and Discretionary Adjustment Procedures 
for Electric Utility Rates, Volume II 


Innovative Approaches to Electric Utility Rate 
Structure: Marginal Costs and Time-Varying Rates, 
Volume I and Usage and Evaluation 


Innovative Approaches to Electric Utility Rate 
Structure: Marginal Costs and Time-Varying Rates, 
Volume II 


The Measurement of Electric Utility Productivity, 
Volume I, Usage of the Work on Electric Utility 
Productivity: An Evaluation 


The Measurement of Electric Utility Productivity, 
Volume II 

Report of Workshops on Four Specific Problems of 
Importance in Government Relations with Small 
Technically Based Companies and Inventor- 
Entrepreneurs - Final Report 
An Evaluation of the Small Business Administratio 
Innovation Loan Program - Final Report 


Author Date NBS No. 
November 
1979 


November 
1979 


November 
1979 


J. W. Wilson & 
Associates, Inc. 


January 
1980 


January 
1980 
American Assoc. October 
of Small Researc 1975 
-Companies 


Moshman Assoc., April 76-11 
lise : 1976 


NTIS No. 
PB80150675 


None 


PB80203177 


None 


None 


PB80208978 


PB80148604 


None 


PB80147333 


PB80151220 


PB253115 


Project 
Number 
80 


85 


87 


78 


Title 


Author 


Date 


NBS No. 


NTIS No. 


Directory of Living Aids for the Disabled Person Veterans | April GPO 982-361-487-3387 
Administration 1982 


Prospects for an OSHA/ETIP Project to Facilitate 
Technological Innovation (see Project No. 157) 


Management of Federal R&D for Commercialization, 
Executive Summary 


Management of Federal R&D for Commercialization, 
Final Report 


Management of Federal R&D for Commercialization, 
Appendices: Supporting Documentation 


78-57, 78-58, and 78-58A 


Subsidies, Capital Formation, and Technology 
Change; Municipal Wastewater Facilities, Vol. l 


Subsidies, Capital Formation, and Technological 
Change; Mass Transit, Vol. 2 


Subsidies, Capital Formation, and Technological 
Change; First Generation Nuclear Power Plants, 
ite Rae 


Subsidies, Capital Formation, and Technological 
Change; Maritime Transport, Vol. 4 


Subsidies, Capital Formation, and Technological 
Change; Primary Scientific and Technical Journal 
Publishing, Vol. 5 


Subsidies, Capital Formation, and Technological 
Change; Health Facilities, Vol. 6 


M. Mulkey, Urban 
Institute 
R. G. Weiss, NBS 


N. M. McEachron 
etrral., SRI 
International 


Charles River 
Associates 


May 
1980 


September 
1978 


July 
1977 


November 
1977 


November 
1977 
December 
1977 
September 


1978 


September 
1978 


PB80179260 


PB292851 


PB292852 


PB292853 


Available as a set 
under PB292850 
PB285288 


PB285289 


PB285290 


PB285291 


PB285292 


PB285293 


Project 


Number 


78 


88 


94 


96 


116 


Title Author 
Subsidies, Capital Formation, and Technological 
Change; Local Air Transport, Vol. 7 


Subsidies, Capital Formation, and Technological 
Change; Summary and Conclusions, Vol. 8 


78-40 through 78-47 


The Evaluability Assessment of the Developing S. Clarren, 

Experiment in Post Marketing Surveillance of Performance 

Prescription Drugs (also see Project 169) Development 
++ Institute 


GuoMe. Britan 


Bureaucracy and Innovation: An Ethnography of 
Policy Change 


A Real-Time Case History of ETIP Project 67-- 
Integrated Utility System Application 


J. G3) Taylor. 
L. C. Lamar 
System Planning 
Corporation 
Logistics 
Management 
Institute 


Life Cycle Costing in the Procurement of Electric 


Ranges 


Life Cycle Costing in the Procurement of 

Refrigerator-Freezers 

Life Cycle Costing in the Procurement of Window Logistics Manage 

Air Conditioners ment Institute 

Assessing the Results of the Portable Oxygen N. B. McEachron 

Generator Experiment . E. G. Woodward 
SRI International 


Assessing the Results of the LCC Water Heater SRI International 
Experiment, Volume 1 


Assessing the Results of the LCC Water Heater SRI International 
Experiment, Volume 2 


Date NBS No. 
January 78-46 
1978 
May 78-47 
1978 
March 81-96 
1981 
January 
1981 
March 78-59 
1978 
March 76-09 
1976 
March 76-10 
1976 . 

July 76-19 
1976 
October 78-50 
1977 


September 18-1 
1977 


September Ros 52 
1977 


NTIS No. 
PB285294 


PB285295 


Available as a set 
under PB285287 
PB81223232 


Sage Library of 
Social Research 
Volume 115 

ISBN 0-8039-1506-3 
PB292770 


PB253488 


PB253260 


PB256642 


PB289886 


PB291326 


PB291827 


++ As this*report was Asie 1 eed Load 8 Reh Rok wal i be published under Project 169: An Experiment. in 
Post Marketing Surveillance of Prescription Drugs: An Initial Status Report, November 1982, 


NES =GCR= EEL. - Sits woot go octane eee ae 


‘ 
3 


Project 
Number 
120 


148 


Title Author Date NBS No. 


Agency Impact Analysis: Life Cycle Costing in J. Lawrence 1979 79-68 
Research Triangle 
Institute 
S. Garrity, NBS 


the Public Building Service 
Timbers & Co. November 71739 
L972 
Kempter-Rossman July 
1976 


International 
Kempter-Rossman July 
International 1976 


Prototype Procurement--A Case Study in 
Experimental Technology Check Wrapping Machines 
of the U.S. Treasury Department 
Value Incentive Program, Household Goods Shipping 
Containers 

Value Incentive Program: Computer Room Air 
Conditioner 


November 
1976 


Value Incentive Program, Energy Conservation 
Compressors for Computer Room Air Conditioning 


October 
1976 


October 77-38 
1977 


1978 78-49 


Value Incentive Program, Reduce Welding/Brazing 
_on Wardrobe Hinge 
Commodity Supply Policies 


Charles River 
Associates 
‘eeMe Utcterback 
A. E. Murray 
Massachusetts 
Institute of 


The Influence of Defense Procurement on the 
Development of the Civilian Electronics Industry 


Evaluating the Impact of Securities Regulation on 
Venture Capital Markets 


J: J. Cordes 
George Washingto 
University 
G. Tassey, NBS 


The Effectiveness of Venture Capital Markets in G. Tassey, NBS 


the U.S. Economy 


September 
1979 


The FCC Public Message Services Policy Change: Joepels 
An ETIP Evaluability Assessment Report, S. Kirby 
Volumes 1 and 2 (also see Project No. 16) S. Watson 
Urban Institute 
R. G. Weiss, NBS 


NTIS No. 
PB81204356 


PB273950 


PB256643 


PB258991 


PB259978 


PB260523 
PB272700 


PB290164 


PB81103582 


Public Policy 


Vole25 ,aNoee 


Falta 
GPO SN-003-003- 
O2ii7=2 


Project 


Number 


ir 


158 


161 


Tree 


Prospects for an OSHA/ETIP Project to Facilitate 
Technological Innovation (also see Project No. 85) 


Balancing the Objectives of Clean Air and 
Economic Growth: Regulated Markets in Emission 
Reductions 


Environmental Regulation in Theory and Practice: 


EPA's Process of Setting Best Pract. Control Tech. 


Standards 


Regulatory Administrative Experiment Manual 
The Regulatory Processes & Effects Project 


An Assessment of an Experiment Accelerating the 
Development of Nuclear Standards (also see 
Project NO. 47) 


An Evaluation Design Concept for Regulatory 
Reform: The Waiver of Efficacy Data 

(also see Project No. 48) 

The FCC Public Message 

Services Policy Change: An ETIP Evaluability 
Assessment, Volumes I and II 


Toward Competitive Provision of Public Message 
Services? (also see Projects 157 and 169) 


— ———— 7 ee sy RE i AI NO TEED A AIRE RN AAT OOS 


Author 
M. A. Mulkey 
Urban Institute 
R. G. Weiss, NBS 


Urban Institute 


Urban Institute 


J. P. Woodward 
Performance 
Development 
Institute 

S. Garrity, NBS 
Arthur=Dvwolatete 


Jeo betissetral. 
Urban Institute 
R. G. Weiss, NBS 


.Performance 


Development 
Institute 


er Sy = 


Date 
May 
1980 


January 
1979 


December 
1978 


February 
belie 


February 
1979 


June 
1980 


May 
19079 


September 
1974 


October 
1981 


NBS No. 
NBS 
Tech- 
aca.) 
Note 
1104 


719-62 


79-63 


79-64 


9=05 


NBSIR 
80-2086 


NTIS No. 


PB80179260 


PB300835 


PB297499 


PB298446 


PB298631 


PB80202104 


PB300846 


GPO SN-003-003- 
02117-2 


ae 
St) SS Ng a A 


Project 
Number 
164 


166 


167 


168 


169 


Deterrence & Demand 


Title 
An International Study of Economic Benefits 
Attributable to R&D, by Source and Sector of 
Performance 


The Solar Energy/Utility Interface, Proceedings 
of Workshops and Conference 


Innovation, Competition, and Government Policy 
in the Semiconductor Industry Industry 


Innovation, Competition, and Government Policy: 
A Framework for Analysis 


Innovation 


Interim Report on Potential ICO/ETIP 
Administrative Experiments (also see 
Project No. 131) 


Incentives for Technological Innovation: 
An ETIP Policy Research Series 


Volume 1, Emission Offset Policy at Work: 
A Summary Analysis of Eight Cases 


Volume 2, Market Mechanisms for Emission 
Regulation & Enforcement of Emission Limits: 


Taxonomy of Incentive Approaches for Stimulating 


Author Date NBS No. NTIS No. 
R. U. Ayres June PB292783 
J. Lewis 1978 


S. D. Collier 
Delta Research 
Corporation 
R. J. Harman, 
Editor 

George Washingto 
Universit 
Charles River 
Associates 


78-60 
December Department of Energy 
1978 HCP/4202-1 

80-91 | PB81105009 

80-92 

August 1o= 39 

1978 


80-85 


PB80226327 


J. Booth PB287932 
Z. Cook 

A. R. Ferguson 
Public Interest 
Economics Center 
M. Mulkey 
W. Frederick 
Performance 
Development 
Institute 

D. Fulmer, NBS 


PB80202575 


80-86 


W. H. Foskett 
Performance 
Development 
Institute 


80-87 


Project 


Number 
169 


Title 
Volume 3, Opportunities for Innovation: 
Administration of Sections 111 (j) and 113 (d) (4) 
of the Clean Air Act & Industry's Development of 
Innovative Control Technology 


Volume 4, Analysis of the Rationale & Public 
Comment Regarding EPA's Proposed Regulation on 
Regional Consistency 


Volume 5, An Annotated Bibliography of Literature 
on Market Mechanisms & Economic Incentives for 
Environmental Regulation 


Volume 6, Innovation by Regulation: The 
Administration of Control Technology Requirements 
Under the Clean Air Act 


Volume 7, Periodic Review and Technological 
Escalation of Performance Standards: New Source 
Performance Standards Review Process 


Volume 8, Controlled Trading and Site Specific 
SIP Revisions: Competing for Attention in 
Crowded Administrative Routes 


The Evaluability Assessment of the Developing 
Experiment in Post-Marketing Surveillance of 


Prescription Drugs (also see Project No. 88) 


Toward Competitive Provision of Public Message 


Services? (also see Project No. 161) 


Author 
J. Evans 
Performance 
Development 
Institute 


J. Evans 

W. H. Foskett 
Performance 
Development 
Institute 


S. Watson 
Performance 
Development 
Institute 


Performance 
Development 
Institute 


Date NBS No. 
April 
1980 


May 
1980 


April 
1980 


March 
1981 


December 
1980 


December 
1980 


March 
1981 


October 
1981 


NTIS No. 
PB80205024 


PB80 203730 


PB80205610 


PB81221616 


PB81218117 


PB81218273 


PB812232 


PB82138819 


0z-¥ 


Project 
Number 


169 


172 
173 


174 


175 


180 


186 


187 


192 


None 


Author 
S. Clarren 
T. Nalley 
C. Zuiches 


Date NBS No. 
Performance 
Deve lopment 


November 82-99 
1982 
Institute 


Corporation 1978 
Z. Cook 1979 


E. M. Graham December 
1978 


Public Interest September 
Subcommittee of 1979 
the Domestic 
Policy Review, 
UxS2- Dept... of 

1979 
October 
1979 


Title 
The Experiment in Post-Marketing Surveillance of 
Prescription Drugs: An Initial Status Report 


Impacts of Government Regulations on the Iron 
and Steel Industry 
An Exploration of Regulatory Incentives for 


Innovation: 6 Case Studies 
Technological Innovation and the U.S. Economy: 
The International Aspects 


Report of the Public Interest Subcommittee in 
Final Report, Advisory Committee on Industrial 
Innovation 


Commerce 
Ce ACE HLLL 
J. M. Utterback, 
Editors 


Technological Innovation for a Dynamic: Economy 


Comparative Growth and Comparative Advantage 


October 
1979 


The Comparative Structure of Growth in the Major 
Developed Capitalist Nations 


A Study of Adversarial and Cooperative Relations 
Between Business and Government in Four Countries 


Study of Strategies for Market Aggregation - J. D. Lewis May fechas 
Final Report H. S. Kleiman i973 

Battelle Columbu 

Laboratories 


NTIS No. 


PB289352 
PB299006 


see Project 180 


GPO 003-000-00553-4 


Permagon Press 
LC#T21.T42 1979 


None 


None 


National Academy 
Press Tas 
University Micro- 
films International 
8113574 

COM 73-11373 


Project 
Number 
None 


None 


None 


None 


None 


None 


None 


None 


None 


Lute Author Date NBS No. 
Study of Strategies for Market Aggregation H. Kleiman May 73-01A 


Appendices J. D. Lewis TOFS 
W. Riddle 
Battelle Columbus 


Laboratories 
W. P. Sommers 
W. Hockersmith 


March 73-02 


was 


July PAF AUTS 
$973 


Government Procurement as an Incentive to 
Commercial Technology and Innovation 


R. W. Shaw 
R. Brudno 
Booz-Allen 


Hamilton 
Forecasting 
International 


Technical Transfer and Technological Innovation: 
Annotated and Selected Bibliography 


Development of Criteria and an Operating P. G. Dickey 
Procedure for Screening Government Owned Patents Ko Do Brown 
Final Report, Volume I R. W. Aaron 

S. A. Lambert 


ITT Research 
Institute 


Volume II 
Proceedings of GSA/ETIP Symposium on Procurement 
Practices, May 29-31, 1974 Supply Service 
ETIP: The First 18 Months, A Progress Report of J. D. Lewis, NBS 
the Experimental Technology Incentives Program, 
National Bureau of Standards 


Proceedings of Procurement Practices Symposium- 
Health Care, October 14-16, 1975 


January 
1975 


February 
1976 


J. G. Berke, NBS 
Editor 

Veterans Admin. 
C. W. N. Thompson 


February 
1976 


February NBSIR 
1977 = 


System Proposal Preparation and Evaluation 
Procedure 

Proceedings of a Symposium: 
Product Improvement 


C. Hulick, NBS 
Editor 

-Federal Supply 
Service 


Communicating for 


~s 
~S 
— 
Nh 
ho 
— 


April 73-07 
1973 
73-07A 


NTIS No. 
COM 75 =11973 


COMW3=11375 


COM 73-11374 


COM 74-10939 


COM 74-10940 
COM 75-10527 


PB246435 


PB254298 


PB273666. 


PB264919 


cate — ¥ 


Project 
Number 
None 


None 


None 
None 


None 


None 


Title 
Innovation in State Public Utility Commissions: 
An Exploratory Study of Techniques in Energy 
Regulation 
Procurement of Evaluation Systems: A Case Study 
of the Parametric Factor Evaluation Approach to 
Source Selection 


Author Date NBS No. 


K. R. Flaherty, June NBSIR 
NBS 1980 80-2046 
A. S. Libman, NBG July NBSIR 

1980 80-209 


Monitoring System Development: A Framework and 
Application 1981 Si=232 
to Monitor Complex System Developments 


Incentives for Technological Innovation in Air 
Pollution Reduction: An ETIP Policy Research 
Series, Volume 9, Internal Offsets 
Final Report - Government and Innovation: 
Experimenting with Change 


M. E. Mogee, NBS | April NBSIR 
1982 82-247 


R. Herbert November 82-100 
H. Hoar 1982 


NTIS No. __ 


None 


PB80204654 


PB82115817 
PB82104308 


PB82208372 


ae 


cr 


u 


WNBS-174A (REV. 2-80) 
ea ae — 


U.S. DEPT. OF COMM. 1. PUBLICATION OR 


BIBLIOGRAPHIC DATA REPORT NO. 
SHEET (See instructions) NBS-GCR-ETIP 82-1400 


4. TITLE AND SUBTITLE 
Government and Innovation: Experimenting with Change 


2. Performing Organ. Report NoJ 3. Publication Date 


December 1982 


(The Final Report of the Experimental Technology Incentives 
Program 


5. AUTHOR(S) 


Richard Hebert and Ralph Hoar 
6. PERFORMING ORGANIZATION (If joint or other than NBS, see instructions) 


| 7. Contract/Grant No. 
NB82NAAM6111 
8. Type of Report & Period Covered 

Final 


NATIONAL BUREAU OF STANDARDS 
DEPARTMENT OF COMMERCE Ralph Hoar 
WASHINGTON, D.C. 20234 


9. SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street, City, 
National Bureau of Standards 

Department of Commerce 

Washington, D.C. 20234 


State, ZIP) 


10. SUPPLEMENTARY NOTES 


[__] Document describes a computer program; SF-185, FIPS Software Summary, is attached. 


11. ABSTRACT (A 200-word or less factual summary of most significant information. If document includes a significant 
bibliography or literature survey, mention it here) 


This final report of the Experimental Technology Incentives 
Program (ETIP) contains a brief history of the program, a 
description of the major projects and accomplishments, a 
listing of lessons learned and a sampling of outside views 

and opinions of the program's strengths and weakmesses. An 
appendix contains a list of all projects (numbered sequentially) 
and published reports. The appendix also describes the process 
by which greater project detail and data can be obtained 


12. KEY WORDS (Six to twelve entries; alphabetical order; capitalize only proper names; and separate key words by semicolons) 


administrative experiments; technology policy; innovation; 
research and development; procurement; economic assistance;regula 10 
13. AVAILABILITY + 714. no. oF 
PRINTED PAGES 
146 


[X] Unlimited 
[_] For Official Distribution, Do Not Release to NTIS 


EB] ales From Superintendent of Documents, U.S. Government Printing Office, Washington, D.C. 


15. Price 


[X] Order From National Technical Information Service (NTIS), Springfield, VA. 22161 $13.50 


USCOMM-DC 6043-P80 


